Joint coding of multiview video and depth data using virtual view synthesis

To compress multiview video and depth information, we synthesize a virtual image for the current view using color and depth data of neighboring views. In this article, we then use a view interpolation prediction scheme at the virtual image to improve the inter-view prediction. We also propose a solution for overlapping regions and empty holes that are generated during the intermediate view synthesis process due to occlusion and disocclusion situations. Experimental results show that the proposed methods achieve approximately 0.65 dB of Peak Signal-to-Noise Ratio (PSNR) gain on average for multiview depth data and 0.17 dB of PSNR gain for multiview video coding, compared with the reference software, Joint Multiview Video Model 1.0. We also show that our method is even more powerful for smaller search ranges. Furthermore, we examine the effects of inter-view prediction on hierarchical Bpictures. VC 2010 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 20, 370– 377, 2010; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI