Dynamic 3-D Scene Analysis Through Synthesis Feedback Control

The analysis of 3D scenes consisting of nonrigid moving objects from 2D image sequences is discussed. A parametric description of dynamic objects is extracted and the time variant scene parameters are estimated throughout the sequence by employing an analysis by synthesis approach. From the parametric scene description images are synthesized and compared with the original input images of the camera. Frame differences between both images are evaluated to estimate a scene parameter update. The analysis system is applied to video phone scenes as a data compression algorithm where the scene parameters are transmitted and the output sequence is synthesized at the receiver. Index Terms – 3D scene analysis, dynamic scene analysis, image sequence analysis, 3D motion estimation, 3D object tracking, 3D shape reconstruction, 3D shape adaptation, analysis by synthesis, object–oriented video phone coding.