Ragdolls in Action – Action Recognition by 3d Pose Recovery from Monocular Video

We present a novel approach to reconstruct and track articulated objects, specifically humans, in 3D from monocular videos for action recognition, by combining techniques from both image processing and 3D computer animation. The goal is to establish a system that is able to recognize basic actions (like walk, run) from frame to frame in a scene with more than one person. In a first step a feature-based detection algorithm extracts hints of body parts from input videos. Second, a virtual world is established in which the extracted 2D measurements are translated into 3D rays. A ragdoll a model used to represent realistic human bodies in 3D animation is attached to a bundle of rays and it adjusts itself with the help of physics simulation. Third, the ragdoll's pose is used to query a motion capture database in order to obtain predictions for the next time frame and to build the basis for action recognition.