The 16th Meeting on Image Recognition and Understanding RGB-D based 3D-Object Recognition by LLC using Depth Spatial Pyramid

Recently, high-accuracy RGB-D cameras are commertially available, which are capable of providing high quality three dimension information (color and depth). In this paper, we propose an object recognition method where the techniques of object recognition in 2D are extended to 3D. Recent image classification systems mainly consist of the following three parts: feature extraction using scaleinvariant feature transform (SIFT), coding scheme using bag-of-features (BoF) and pooling process using spatial pyramid matching (SPM) [1]. The SPM is also regarded as an extension of BoF, which partitions the image into hierarchical spatial sub-regions and computes histograms of local features from each sub-region. This spatial pyramid restricted by position has shown very promising performance on many image classification tasks. These techniques used for 2D images are applied to 3D object recognition without any changes so far. For that reason, even though the depth information captures the overall shape of an object, conventional methods use depth information only to extract the local feature. In our proposed approach, the overall object shape is captured by the depth spatial pyramid based on depth information. In more detail, multiple features within each subregion of the depth spatial pyramid are pooled. As a result, the feature representation including the depth topological information is constructed. We use not only SIFT, but also histograms of oriented normal vectors (HONV [2]) for the depth image, which are originally designed to capture local geometric characteristics. We also adopt localityconstrained linear coding (LLC [4]), which utilizes local constraints to project each descriptor into its local-coordinate system.