Technical Report 2003-02-CIRL-CS-JHU Stereo-Based Direct Surface Tracking with Deformable Parametric Models

We present a general set of techniques for directly estimating and tracking deformable surfaces from a stream of rectified stereo pairs. These techniques are based on iterative updating of surface representations directly from image information and use no disparity search except during initialization. We perform the tracking through an iteratively re-weighted least squares minimization wherein a mask is incorporated to increase robustness to occlusion. The algorithms are formulated for a general family of linear in parameters surface models, and discussed for the specific case of tensor product surfaces. We also show that it is possible to naturally incorporate dynamic surface segmentation into the algorithm. These algorithms have been implemented and run at or near frame rate. We evaluate their performance on a variety of image data, including cloth and bending paper.