Learning body part‐based pose lexicons for semantic action recognition

Semantic action recognition aims to classify actions based on the associated semantics, which can be applied in video captioning and human-machine interaction. In this paper problem is addressed by jointly learning multiple pose lexicons body parts. Specifically, visual models are learnt, one model with part, characterises likelihood of an observed frame being generated from hidden poses. Moreover, lexicon simultaneously learnt along models. One part that establishes a probabilistic mapping between poses semantic parsed textual instructions. To capture temporal relations among parts, transition also measure probability alignment transitioned position another position. The part-based provides novel method cross-modality correlation, other spatial data. Action classification finally formulated as finding maximum posterior given sequences frames follow poses, subject most likely sequences. Experiments were conducted five datasets validate effectiveness proposed method.