Contextual Pooling in Image Classification

The original bag-of-words (BoW) model in terms of image classification treats each local feature independently, and thus ignores the spatial relationships between a feature and its neighboring features, namely, the feature’s context. However, our intuition and empirical studies tell the importance of such spatial information. Although the global spatial information can be captured with the spatial pyramid matching scheme, the subject of capturing local spatial relationships between features is still open. In this paper, we propose a new method to embed such local spatial (context) information into the BoW model. A vector reflecting context information is firstly extracted along with each feature, context patterns are then code-specifically trained, and thus the context information is elegantly embedded into the BoW model by contextual pooling according to different context patterns. Extensive experiments on the PASCAL VOC 2007 dataset show that our method greatly enhances the BoW model, and achieves the state-of-the-art performance.