Flight Testing of a Fault-Tolerant Control and Vision-based Obstacle Avoidance System for UAVs

The paper describes the development, implementation and flight testing of the Visual Threat Awareness (VISTA) system and the Multi-layer Architecture for Trajectory Replanning and Intelligent plan eXecution (MATRIX) for autonomous intelligent control of Unmanned Aerial Vehicles (UAVs). The VISTA system generates information on the threats and obstacles in real-time, and passes it on to the MATRIX system that makes mission-related decisions and generates new waypoints and a trajectory that safely avoids the obstacle. The VISTA system combines binocular visual stereo, perceptual organization, graph partitioning and feature tracking for a passive system to enable real-time obstacle detection. Computational stereo performance has progressed such that there now exist several commercial or open source implementations that operate at frame rate, but suffer from well known correspondence errors. We show that introducing a global segmentation step after commodity stereo can increase robustness and leverage existing stereo software. The global segmentation step is based on a graph structure appropriate for collision detection, human vision inspired perceptual organization and graph partitioning using the minimum s-t graph cut. This system has been prototyped using Sarnoff Corp’s Acadia I vision processor to enable 640x480@(3-5) Hz operation on embedded avionics at accuracies of 6ft at 50ft. We describe VISTA system theory and show proof of concept and flight experiment results of the MATRIX/VISTA system on Georgia Tech’s GT-Max autonomous helicopter. This work was supported under a DARPA IXO Software Enabled Control (SEC) program grant (DAAH01-00-C-R187). The authors wish to thank Dr. John Bay for his suggestions and contributions.