Recognition of Arm Movements

This paper describes a method for detection, tracking and recognition of lower arm and hand movements from color video sequences using a linguistic approach driven by motion analysis and clustering techniques. The novelty of our method comes from (i) automatic arm detection, without any manual initialization, foreground or background modeling, (ii) gesture representation at different levels of abstraction using a linguistic approach based upon signal to symbolic mappings, and (iii) robust matching for gesture recognition using the weighted largest common sequence (of symbols). Learning Vector Quantization abstracts the affine motion parameters as morphological primitive units, i.e., ”letters”, clustering techniques derive sequence of letters as ”words” for both subactivities and the transition occurring between them, and finally the arm activities are recognized in terms of sequences of some subactivities. Using activity cycles from six kinds of arm movements, e.g., (slow and fast pounding), striking, swing, swirl, and stirring, which were not available during training, the performance achieved is perfect 100% if one allows, as it should be the case for invariance purposes, to recognize slow and fast pounding video sequences as one and the same type of activity.