Terrain Geometry from Monocular Image Sequences

Terrain reconstruction from images is an ill-posed, yet commonly desired Structure from Motion task when compositing visual effects into live-action photography. These surfaces are required for choreography of a scene, casting physically accurate shadows of CG elements, and occlusions. We present a novel framework for generating the geometry of landscapes from extremely noisy point cloud datasets obtained via limited resolution techniques, particularly optical flow based vision algorithms applied to live-action video plates. Our contribution is a new statistical approach to remove erroneous tracks (‘outliers’) by employing a unique combination of well established techniques—including Gaussian Mixture Models (GMMs) for robust parameter estimation and Radial Basis Functions (RBFs) for scattered data interpolation—to exploit the natural constraints of this problem. Our algorithm offsets the tremendously laborious task of modeling these landscapes by hand, automatically generating a visually consistent, view dependent, thin-shell surface mesh within seconds for a typical tracking shot.