Collective Activity Localization with Contextual Spatial Pyramid

In this paper, we propose an activity localization method with contextual information of person relationships. Activity localization is a task to determine “who participates to an activity group”, such as detecting “walking in a group” or “talking in a group”. Usage of contextual information has been providing promising results in the previous activity recognition methods, however, the contextual information has been limited to the local information extracted from one person or only two people relationship. We propose a new context descriptor named “contextual spatial pyramid model (CSPM)”, which represents the global relationships extracted from the whole of activities in single images. CSPM encodes useful relationships for activity localization, such as “facing each other”. The experimental result shows CSPM improve activity localization performance, therefore CSPM provides strong contextual cues for activity recognition in complex scenes.