Virtual view synthesis method and self-evaluation metrics for free viewpoint television and 3D video

Virtual view synthesis is one of the most important techniques to realize free viewpoint television and three-dimensional (3D) video. In this article, we propose a view synthesis method to generate high-quality intermediate views in such applications and new evaluation metrics named as spatial peak signal-to-noise ratio and temporal peak signal-to-noise ratio to measure spatial and temporal consistency, respectively. The proposed view synthesis method consists of five major steps: depth preprocessing, depth-based 3D warping, depth-based histogram matching, base plus assistant view blending, and depth-based hole-filling. The efficiency of the proposed view synthesis method has been verified by evaluating the quality of synthesized images with various metrics such as peak signal-to-noise ratio, structural similarity, discrete cosine transform (DCT)-based video quality metric, and the newly proposed metrics. We have also confirmed that the synthesized images are objectively and subjectively natural. VC 2010 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 20, 378–390, 2010; Published online in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/ima.20253