People, when processing human-to-human communication, utilize everything they can in order to understand that communication, including speech and information such as the time and location of an interlocutor’s gesture and gaze. Speech and gesture are known to exhibit a synchronous relationship in human communication; however, the precise nature of that relationship requires further investigation...

Human communication is a complex multimodal behavior that is deeply embedded within our environment. From narrative monologues (Dale, 2014) to dyadic task performance (Paxton, Abney, Kello, & Dale, 2014), recent efforts have sought to identify multimodal signatures of different types of communication. We extend these efforts in the current project by investigating the multimodal signatures of l...

Multimodal communication involves multiple communicative channels including speech, facial movement, and gesture. Relatively few studies on how various communicative modalities are aligned in natural, face-to-face communication exist. As part of a larger project, the current study investigates how discourse structure, speech features, eye gaze, and facial movements interrelate during a map coor...

This contribution focuses on multimodal interaction techniques for a mobile communication and assistance system on a robot platform. The system comprises of acoustic, visual and haptic input modalities. Feedback is given to the user by a graphical user interface and a speech synthesis system. By this, multimodal and natural communication with the robot system is possible.

One of the most common tasks of a robot companion in the home is communication. In order to initiate an information exchangewith its human partner, the robot needs to attract the attention of the human. This paper presents results of two user studies (N = 12) to evaluate the effectiveness of unimodal and multimodal communication cues for attracting attention. Results showed that unimodal commun...