Combining stereo and visual hull information for on-line reconstruction and rendering of dynamic scenes

In this paper, we present a novel system which combines depth-from-stereo and visual hull reconstruction for acquiring dynamic real-world scenes at interactive rates. First, we use the silhouettes from multiple views to construct a polyhedral visual hull as an initial estimate of the object in the scene. The visual hull is then used to limit the disparity range during depth-fromstereo computation. The restricted search range improves both speed and quality of the stereo reconstruction. In return, stereo information can compensate for some of the inherent drawbacks of the visual hull method, such as inability to reconstruct surface details and concave regions. Our system achieves a reconstruction frame rate of 4 fps.