Cross-spectral visual simultaneous localization and mapping (SLAM) with sensor handover

In this work, we examine the classic problem of robot navigation via visual simultaneous localization and mapping (SLAM), but introducing the concept of dual optical and thermal (cross-spectral) sensing with the addition of sensor handover from one to the other. In our approach we use a novel combination of two primary sensors: co-registered optical and thermal cameras.Mobile robot navigation is driven by two simultaneous camera images from the environment over which feature points are extracted andmatched between successive frames. A bearing-only visual SLAM approach is then implemented using successive feature point observations to identify and track environment landmarks using an extended Kalman filter (EKF). Six-degree-of-freedommobile robot and environment landmark positions are managed by the EKF approach illustrated using optical, thermal and combined optical/thermal features in addition to handover from one sensor to another. Sensor handover is primarily targeted at a continuous SLAM operation during varying illumination conditions (e.g., changing from night to day). The final methodology is tested in outdoor environments with variation in the light conditions and robot trajectories producing results that illustrate that the additional use of a thermal sensor improves the accuracy of landmark detection and that the sensor handover is viable for solving the SLAM problem using this sensor combination. © 2012 Elsevier B.V. All rights reserved.