Development and experimentation with synthetic visible speech

Speech perception and speech production are skills that rival other impressive human achievements. Even after decades of intense effort, speech recognition by machine remains far inferior to that of human performance. Also, as we have all experienced, speech synthesis has not yet obtained the naturalness and quality of the speech of even a young child. Our research has led us to the view that speech perception is easy for human beings because multiple sources of information support the identification and interpretation of the language input. Therefore, it is important to define the sources of information and to determine how they are evaluated and integrated to achieve recognition. The paradigm that we have developed permits us to determine which of the many potentially functional cues are actually used by human observers. The systematic variation of properties of the speech signal, combined with quantitative tests of models based on different sources of information, enables the investigator to test the psychological validity of different cues. Thus, our research strategy not only addresses how different sources of information are evaluated and integrated, it can uncover what sources of information are actually used. We believe that the research paradigm confronts both the important psychophysical question of the nature of information and the process question of how the information is transformed and mapped into behavior. Valuable and effective information is afforded by a view of the speaker's face in speech perception and recognition by humans. Visible speech is particularly effective