We address the problem of estimating camera pose relative to a known scene, given a single RGB image. We extend recent advances in scene coordinate regression forests for camera relocalization in RGB-D images to use RGB features, enabling camera relocalization from a single RGB image. Furthermore, we integrate random RGB features and sparse feature matching in an efficient and accurate way, bro...

RGB-D cameras (such as the Microsoft Kinect) are novel sensing systems that capture RGB images along with per-pixel depth information. In this paper we investigate how such cameras can be used for building dense 3D maps of indoor environments. Such maps have applications in robot navigation, manipulation, semantic mapping, and telepresence. We present RGB-D Mapping, a full 3D mapping system tha...

A pixel-level fusion technique for RGB representation of multispectral images is proposed. The technique results in highly correlated RGB components, a fact which occurs in natural colour images and is strictly related to the colour perception attributes of the human eye. Accordingly, specific properties for the covariance matrix of the final RGB image are demanded. Mutual information is employ...

Access to large, diverse RGB-D datasets is critical for training RGB-D scene understanding algorithms. However, existing datasets still cover only a limited number of views or a restricted scale of spaces. In this paper, we introduce Matterport3D, a large-scale RGB-D dataset containing 10,800 panoramic views from 194,400 RGB-D images of 90 building-scale scenes. Annotations are provided with su...

With the increased availability of cheap and reliable depth sensors, imaging systems can now use depth information to better detect and locate objects in a scene. New augmented reality (AR) systems, such as the Microsoft HoloLens, are now mounting depth sensors on their glasses to improve functionality. The motivation behind our project is to perform object recognition using RGB-D (color and de...

Current RGB-D scene recognition approaches often train two standalone backbones for RGB and depth modalities with the same Places or ImageNet pre-training. However, pre-trained network is still biased by RGB-based models which may result in a suboptimal solution. In this paper, we present single-model self-supervised hybrid pre-training framework modalities, termed as CoMAE. Our CoMAE presents ...

In this paper, we present a generalizable model-free 6-DoF object pose estimator called Gen6D. Existing estimators either need the high-quality models or require additional depth maps masks in test time, which significantly limits their application scope. contrast, our only requires some posed images of unseen and is able to accurately predict poses arbitrary environments. Gen6D consists an det...

In this paper, we propose a saliency detection model for RGB-D images based on the contrasting features of colour and depth with a generative mixture model. The depth feature map is extracted based on superpixel contrast computation with spatial priors. We model the depth saliency map by approximating the density of depth-based contrast features using a Gaussian distribution. Similar to the dep...