An Evaluation on Color Invariant Based Local Spatiotemporal Features for Action Recognition

Despite recent advances in the design of features to improve automated human action recognition, color information has so far been overlooked. Nevertheless, color has been proven an important element to the success of automated recognition of objects/scenes and segmentation. For object and scene recognition in static images, robustness to photometric variations has been achieved by describing local regions of spatial interest points in terms of color invariance properties. Such robustness was built on the dichromatic refection model proposed by Shafer. Thus, we extended the space-time interest point detector to incorporate color invariance properties in the feature extraction procedure. We were certain that color could contribute to the distictiveness of some classes. Additionally, in some cases, objects of interest are exhibited in a way that color appears as an essential element in describing the event. We evaluate the performance of the family of color-based STIPs in different application contexts: human actions from movies, violence and pornography. In the former, accuracy rates were improved in more than half of the action classes. In the pornography case, we found that the proposals are the best to increase reliability in critical applications such as digital forensics. Keywords-color invariants; spatiotemporal features; violence detection; pornography detection; human actions