Within the area of advanced man-machine interaction, speech communication has always played a major role for several decades. The idea of replacing the convential input devices such as buttons and keyboard by voice control and thus increasing the comfort and the input speed considerably, seems that much attractive, that even the quite slow progress of speech technology during those decades coul...

In recent years, we have developed a framework of human-computer interaction that offers recognition of various communication modalities including speech, lip movement, facial expression, handwriting/drawing, gesture, text and visual symbols. The framework allows the rapid construction of a multimodal, multi-device, and multi-user communication system within crisis management. This paper report...

In this article, we present the spatial logistics task (SLOT) platform for investigating multimodal communication between 2 human participants Presented are the SLOT communication task and the software and hardware that has been developed to run SLOT experiments and record the participants' multimodal behavior. SLOT offers a high level of flexibility in varying the context of the communication ...

In human communication, utterances are expressed with some structural events, e.g., sentence, speech repairs, control of floor, and etc. These structural events bring important information and quite helpful for a better understanding of human communication. Meanwhile, the human communication is also full of multimodal behaviors, e.g., gesture, gaze, and etc. As non-verbal signals, gesture and g...

One major goal of human computer interfaces is to simplify the communication task. Traditionally, users have been restricted to the language of computers for this task. With the emerging of the graphical and multimodal interfaces the effort required for working with a computer is decreasing. However, the problem of communication is still present, and users continue caring about the communicatio...

Multimodal interface aims at overcoming the shortcomings of conventional human-computer communication -serialized input, excessive accuracy, and low input efficiency -by exploiting the complementary nature of multiple modalities in interpreting user intent, thus improving the efficiency and naturalness of humancomputer interaction. In this paper, we present motivations driving multimodal interf...

In interpersonal interaction, the terms synchrony or alignment refer to the way in which communication channels like speech or body movement become intertwined over time, both across interlocutors and within a single individual. A recent trend in alignment research has targeted multimodal alignment, exploring how various communication channels affect one another over time (e.g., Louwerse et al....

This paper presents a new multimodal system for group dynamics and interaction analysis. The framework is composed of a mic array and multiview video cameras placed on a digital signage display which serves as a support for interaction. We show that visual information processing can be used to localize nonverbal communication events and synchronized with audio information. Our contribution is t...

In this work, we propose a multimodal interaction framework for robust human-multirobot communication in outdoor environments. In these scenarios, several human or environmental factors can cause errors, noise and wrong interpretations of the commands. The main goal of this work is to improve the robustness of human-robot interaction systems in similar situations. In particular, we propose a mu...