Exemplar-based Action Recognition in Video

Over recent years, a lot of progress has been made towards automatic annotation of video material, especially in the context of object and scene recognition. However, in comparison, action recognition is still in its infancy. Whereas originally silhouette-based approaches or approaches based on pose estimation have been studied mostly, good results have been reported recently using extensions of traditional object recognition approaches to the spatio-temporal domain [2, 3, 5]. These methods consider actions as typical spatio-temporal patterns that can be modeled using local features, optical flow, or gradient-based descriptors. It is in this line of research that our work is situated. More specifically, we build on the work of Chum et al. [1] which is an exemplar-based approach for object detection using local features that can be situated somewhere in between sliding window based approaches and the Implicit Shape Model (ISM) [4]. We extend the exemplar-based object detection work of Chum et al. [1] to the spatio-temporal domain where we use the recently proposed local, dense, scale-invariant spatio-temporal features [6]. The overall pipeline is shown in figure 2.