Complex Continuous Meaningful Humanoid Interaction: A Multi Sensory-Cue Based Approach

Human interaction involves a number of factors. One key and noticeable factor is the mass perceptual problem. Humans are equipped with a large number of receptors, equipped for seeing, hearing and touching, to name just a few. These stimuli bombard us continuously, often not on a singular basis. Typically multiple stimuli are activated at once, and in responding to these stimuli, variations of responses are exhibited. The current aim of our project is to provide an architecture, that will enable a humanoid robot to yield meaningful responses to complex and continuous interactions, similiar to that of humans. In this paper we present our humanoid, a system which is able to simultaneously detect the spatial orientation of a sound source, and is also able to detect and mimic the motion of the upper body of a person. The motion produced by our system is human like – ballistic motion. The focus of this paper is on how we have come about the integration of these components. A continuous interactive experiment is presented in demonstrating our initial effort. The demonstration will be in the context of our humanoid interacting with a person. Through the use of spatial hearing and multiple visual cues, the system is able to track a person, while mimicking the persons upper body motion. The system has shown to be robust and tolerable to failure, in performing experiments for a long duration of time. Currently supported by the Science and Technology Agency(STA) of Japan, as a STA Fellow.