View invariant identification of pose sequences for action recognition

In this paper, we represent human actions as short sequences of atomic body poses. The knowledge of body pose is stored only implicitly as a set of silhouettes seen from multiple viewpoints; no explicit 3D poses or body models are used, and individual body parts are not identified. Actions and their constituent atomic poses are extracted from a set of multiview multiperson video sequences by an automatic keyframe selection process, and are used to construct a Hidden Markov Model. Given new single viewpoint sequences, we can recognize key pose sequences and changes in viewpoint simultaneously. Recognized pose sequences can potentially be parsed to find actions, if the actions are described by a grammar in which atomic pose pairs act as terminal symbols. We provide experimental results under constant viewpoint and changing viewpoint conditions.