Factorization-based Non-rigid Shape Modeling and Tracking in Stereo-Motion

In recent years, researchers have tackled the topic of augmenting “structure from motion” with stereo information. Unfortunately nearly all stereo-motion algorithms assume that the scene is rigid. In [16] the factorization-based structurefrom-motion method for rigid objects was proposed, and soon it was extended to non-rigid or deformable objects in [17]. In this paper we propose a framework of factorization-based non-rigid shape modeling and tracking in stereo-motion. We construct a measurement matrix consisting of the stereo-motion data captured from a stereo-rig. Organized in a particular way this matrix could be factored by Singular Value Decomposition (SVD) to get the 3D basis shapes, their configuration weights, rigid motion and camera geometry. With this, the stereo correspondences can be inferred from motion correspondences only requiring that a minimum of 3K point stereo correspondences (where K is the dimension of shape basis space) are determined in advance. Basically this framework still keeps the usage potential of rank constraints, meanwhile it has other advantages such as simpler correspondence and accurate reconstruction even with short image sequences. Results of synthesized data and real stereo sequences are given to demonstrate the performance.