Virtual View Generation for 3D Digital Video

systems use digital models to provide interactive viewing. We present a 3D digital video system that attempts to provide the same capabilities for actual performances such as dancing. Recreating the original dynamic scene in 3D, the system allows photorealistic interactive playback from arbitrary viewpoints using video streams of a given scene from multiple perspectives. V irtual reality can become a medium of great usefulness in entertainment and education if it can incorporate recordings of actual events. While much work has gone into creating synthetic environments that correspond to counterparts in the real world, few have attempted to incorporate real people and events into such environments. To achieve this, an event of interest must be truthfully captured by real-time sensors such as video. From this recording, faithful digital replications must then be created such that the original performances can be presented using standard computer graphics methods and viewed from arbitrary perspectives. We present a system for generating and replaying photorealistic 3D digital video sequences of real events and performances. 3D video embodies the truthfulness of video recordings and the inter-activity of 3D graphics (see Figure 1). This system employs Multiple Perspective Interactive Video (MPI-Video), 1 an infrastructure for the analysis and management of, and interactive access to, multiple video cameras monitoring a dynamically evolving scene such as a football game. The problem of virtual view creation, or view synthesis or interpolation of real scenes, has received increasing attention in recent years. Current approaches divide into two classes: image-based and model-based. Image-domain methods employ warping or morphing techniques to interpolate intermediate views from real images. Model-based methods first recover the geometry of the real scene; the resulting 3D model can then be rendered from desired viewpoints. Our method belongs to the latter class. Image-based methods The best-known image-domain method is Apple's QuickTime VR. 2 By capturing the 360-degree views (cylindrical panoramic images) of an environment from a fixed position, you can interactively adjust view orientation by rendering the corresponding portion of the panorama. Other approaches use image warping. Chen and Williams 3 determined camera transformations with pixel correspondences, then used morphing to generate intermediate views. Skerjanc and Liu 4 used known camera positions to obtain depth information and generate virtual views. Chang and Zakhor 5 obtained depth information by using an uncalibrated camera that " scans " a stationary scene and transforms points on camera image planes onto the plane of the …