Multimodal Input Fusion in Human-computer Interaction
نویسنده
چکیده
In this paper, we address the modality integration issue on the example of a system that aims at enabling users to combine their speech and 2D gestures when interacting with life-like characters in an educative game context. In a preliminary limited fashion, we investigate and present the use of combined input speech, 2D gesture and environment entities for user system interaction.
منابع مشابه
Multimodal Fusion for Interaction Systems
Researchers in computer science and computer engineering devote now a significant part of their efforts in communication and interaction between human and machine. Indeed, with the advent of real-time multimodal and multimedia processing, computer is no longer seen only as a calculation tool, but as a machine of communication processing, a machine that accompanies, assists or promotes many acti...
متن کاملGeneric Pipelined Multi-Agents Architecture for Multimedia Multimodal Software Environments
Multimodal human-computer interaction needs intelligent architectures in order to enhance the flexibility and naturelness of the user interface. These architectures have the ability to manage several multithreaded input signals from different input media in order to perform their fusion into intelligent commands. In this paper, a generic comprehensive agent-based architecture for multimodal eng...
متن کاملSemantic Based Information Fusion in a Multimodal Interface
The fusion of different input channels is a key factor for multimodal human-computer interfaces. In this work, we present a new concept for modality fusion. Our aim was an interface architecture that is restricted neither to a specific application field nor to specific input commands. We use a semantic network in combination with a dictionary for describing the possible user inputs – the intera...
متن کاملConcept-based evidential reasoning for multimodal fusion in human-computer interaction
Fusion of information from multiple modalities in Human Computer Interfaces (HCI) has gained a lot of attention in recent years, and has far reaching implications in many areas of human-machine interaction. However, a major limitation of current HCI fusion systems is that the fusion process tends to ignore the semantic nature of modalities, which may reinforce, complement or contradict each oth...
متن کاملModality Fusion in a Route Navigation System
In this paper we present the MUMS Multimodal Route Navigation System which combines speech, pen, and graphics into a PDA-based multimodal system. We focus especially on the three-level modality fusion component which we believe provides an accurate and more flexible input fusion than the usual twolevel approaches. The modular architecture of the system supports flexible component management, an...
متن کامل