3D Reconstruction from Stereo Camera Dynamic Image based on Particle Filter

Abstract: A new method for 3D reconstruction from dynamic stereo image by using state estimation with particle filter is proposed. Associations of feature points between two images and 3D position of the feature points are estimated simultaneously by particle filter. It is due to the applicability of particle filter for nonlinear and non-Gaussian state space model including unknown associations, non-Gaussian distributions appeared here, and nonlinearlity of projection. We assume that there are missing and error detection of feature points through the image processing for feature extraction. The novelty of our method is simultaneous estimation of the unknown associations and the 3D positions of feature points while most conventional methods are using 2 step estimation, i.e., firstly estimate the associations then secondly calculate the 3D positions depending on the estimated associations. Further improvement of estimation called Rao-blackwellization, which is a method for variance reduction of the estimate, is used at the implementation of particle filter with extended Kalman filter for nonlinear part of the model. Simulation experiment illustrates the efficiency of the method. At the concluding part, we mention about a possibility of this method to provide a basis for multiple sensor fusion problem in dynamic situation by extending the model into a general form.