Use of Nonverbal Communication in Dialog System

Understanding human emotions and their nonverbal messages is one of the most necessary and important skills for making the next generation of human-computer interfaces (HCI) easier, more natural and effective. Indeed, the first step toward an automatic emotion sensitive human-computer system having the ability to automatically detect users’ nonverbal signals is the development of an accurate and real-time automatic NVC analyzer. Such an analyzer must deal mainly with users’ facial expressions and paralanguage. The main goal of this paper is to compare different methods to combine the results of both classifiers. A prototype of the dialog system was developed in the Department of Computer Science . The proposed system is fully automatic, user-independent and real-time working. Several experiments show that the speech recognition quality is increased by using nonverbal information. Key-Words: Dialog system, nonverbal communication, prosody, face detection and localization, feature extraction and classification.