3D structure from video streams with partially overlapping images

The majority of methods available to recover 3D structure from video assume that a set of feature points are tracked across a large number of frames. This is not always possible in real videos because the images overlap only partially, due to the occlusion and the limited field of view. This paper describes a new method to recover 3D structure from videos with partially overlapping views. The well known factorization method [1] recovers 3D rigid structure by factoring an observation matrix that collects trajectories of feature points. We extend this method to the more challenging scenario of observing incomplete trajectories. This way, we accommodate not only the features that disappear, but also features that, although not visible in the first image, become available later. Under this scenario, the observation matrix has missing entries. We develop three new algorithms to factor out matrices with missing data. Experiments with synthetic data and real video images demonstrate the viability of our approach to recover 3D structure.