Motion, Stereo and Color Analysis for Dynamic Virtual Environments

We develop a vision system for highly mobile autonomous agents that is capable of dynamic obstacle avoidance and active perception. We demonstrate the robust performance of the system in artificial animals with directable, foveated eyes, situated in physics-based virtual worlds. Through active perception, each agent controls its eyes and body by continuously analyzing photorealistic binocular retinal image streams. The vision system estimates optical flow, computes stereo disparity and segments looming targets in the low-resolution visual periphery while controlling eye movements to track an object fixated in the high-resolution fovea. It matches segmented targets against mental models of colored objects of interest in order to decide whether the segmented objects are harmless or represent dangerous obstacles. The latter are localized, enabling the artificial animal to exercise the sensorimotor control necessary to support complex behaviors, such as predation, and obstacle avoidance.