Temporally Coherent 3D Animation Reconstruction from RGB-D Video Data
نویسندگان
چکیده
We present a new method to reconstruct a temporally coherent 3D animation from single or multi-view RGB-D video data using unbiased feature point sampling. Given RGB-D video data, in form of a 3D point cloud sequence, our method first extracts feature points using both color and depth information. In the subsequent steps, these feature points are used to match two 3D point clouds in consecutive frames independent of their resolution. Our new motion vectors based dynamic alignement method then fully reconstruct a spatio-temporally coherent 3D animation. We perform extensive quantitative validation using novel error functions to analyze the results. We show that despite the limiting factors of temporal and spatial noise associated to RGB-D data, it is possible to extract temporal coherence to faithfully reconstruct a temporally coherent 3D animation from RGB-D video data. Keywords—3D video, 3D animation, RGB-D video, Temporally Coherent 3D Animation.
منابع مشابه
A System for 3D Video Acquisition and Spatio-Temporally Coherent 3D Animation Reconstruction using Multiple RGB-D Cameras
We present a system for acquiring synchronized multi-view color and depth (RGB-D) data using multiple off-the-shelf Microsoft Kinect and a new method for reconstructing spatiotemporally coherent 3D animation from time-varying dynamic RGB-D data. Our acquisition system is independent of any specific hardware component for the synchronization of the camera system. We show that the data acquired b...
متن کاملHand Gesture Recognition from RGB-D Data using 2D and 3D Convolutional Neural Networks: a comparative study
Despite considerable enhances in recognizing hand gestures from still images, there are still many challenges in the classification of hand gestures in videos. The latter comes with more challenges, including higher computational complexity and arduous task of representing temporal features. Hand movement dynamics, represented by temporal features, have to be extracted by analyzing the total fr...
متن کاملStructure-Aware and Temporally Coherent 3D Human Pose Estimation
Deep learning methods for 3D human pose estimation from RGB images require a huge amount of domain-specific labeled data for good in-the-wild performance. However, obtaining annotated 3D pose data requires a complex motion capture setup which is generally limited to controlled settings. We propose a semi-supervised learning method using a structure-aware loss function which is able to utilize a...
متن کاملTemporally Consistent Motion Segmentation from RGB-D Video
We present a method for temporally consistent motion segmentation from RGB-D videos assuming a piecewise rigid motion model. We formulate global energies over entire RGB-D sequences in terms of the segmentation of each frame into a number of objects, and the rigid motion of each object through the sequence. We develop a novel initialization procedure that clusters feature tracks obtained from t...
متن کامل3D reconstruction from RGB and Depth Video
3D reconstruction has come a long way since the first attempts more than three decades ago. A variety of new algorithms have been proposed in the literature to solve various aspects of this complex problem. There are many different applications of 3D reconstruction with very diverse methodologies and goals. Our approach, however, is focused on reconstructing rigid objects from RGB and depth vid...
متن کامل