A communicative robot “BUGNOID” that integrates various sensory data and behavior modules is introduced with some experimental results. For the achievement of flexible communication with humans, the robot has multimodal interface holding diverse channels of communication. Moreover, the robot can recognize the environment taking a human’s behavior into account in order to co-exist with them.

Temporal structures such as utterance pauses/overlaps, and duration of facial expression play a crucial roles in realizing smooth and natural human communication. We introduce a novel method for modeling the human communication dynamics based on interval-based representations via hybrid dynamical systems and show several examples to demonstrate how this method captures the characteristics of te...

Mobile broadband internet access and powerful mobile devices make interesting and novel communication applications possible (e.g., recently emerging VoIP applications). Additionally, speech recognition has matured to the point that companies can seriously consider its use. We developed a distributed framework that enables multimodal user interfaces with speech recognition (dictation and command...

In recent years there has been an increasing interest concerning the integration of speech and other communication media in conversational systems, originating the so-called multimodal conversational systems. When several interaction modalities are available, users can receive more feedback from a conversational system and it can also receive more information from users, leading to a reduction ...

Human communication is inherently multimodal. In this study we focus on three channels of spoken discourse: the verbal component, prosody, and gesticulation. We address the question of units that can be identified within these components and in spoken multimodal discourse as a whole. The basic unit of the verbal channel is the clause, reporting an event or a state. A set of prosodic criteria he...

In this work, we present our unified framework of proxemics and multimodal communication based on static data-driven models of how social signals (speech and gesture) are produced and perceived by both a social robot and a human user, and propose extensions to enable dynamic model adaptation of these human communication parameters.