Audiovisual documents provide a wide range of content description through more descriptors from different media types. Indeed, the extraction of these descriptions has received an increasing attention. But, the lack of semantic description always persists. In fact, this lack affects the retrieval process. To address this problem, this paper describes an automatic and semantic description of cin...

Practicing a musical instrument is a rich multisensory experience involving the integration of visual, auditory, and tactile inputs with motor responses. This combined psychophysics-fMRI study used the musician's brain to investigate how sensory-motor experience molds temporal binding of auditory and visual signals. Behaviorally, musicians exhibited a narrower temporal integration window than n...

Humans constantly receive sensory input from several sensory modalities. Via the process of multisensory integration, this input is often integrated into a unitary percept. Researchers have investigated several factors that could affect the process of multisensory integration. However, in this field of research, social factors (i.e., whether a task is performed alone or jointly) have been widel...

In the present study, we examined the distribution and processing of information over time in auditory and visual speech as it is used in unimodal and bimodal word recognition. English consonant-vowel-consonant words representing all possible initial consonants were presented as auditory, visual, or audiovisual speech in a gating task. The distribution of information over time varied across and...

One of the research challenges for future search engines concerns the integration of multimodal and cross-modal, nonverbal, full-body, affective, social, and enactive interaction in the process of search and retrieval of audiovisual content. The paper gives a short presentation of the three-year EU project I-SEARCH (EU 7FP ICT STREP), aiming at creating a novel unified framework for multimodal ...

Combining information from the visual and auditory senses can greatly enhance intelligibility of natural speech. Integration of audiovisual speech signals is robust even when temporal offsets are present between the component signals. In the present study, we characterized the temporal integration window for speech and nonspeech stimuli with similar spectrotemporal structure to investigate to w...

The audiovisual appearance of computer and video games is varied. Still, the interplay of sounds and images in games has not been studied in any rigorous way. In this paper the concept of audiovisual style is introduced to grasp these variations, and categorize existing games into three styles. They are called photorealism, caricaturism, and abstractionism. Moreover, elements that make up the a...

The special issue collects a subset of the best papers presented at the ICRA’05 workshop on the Social Mechanisms of Robot Programming by Demonstration. The workshop took place on April 21st 2005 in Barcelona, Spain, at the IEEE Conference on Robotics and Automation (ICRA’05). Eleven of the best papers presented at the workshop were subsequently submitted for peer review before publication in t...

Speech researchers have long been interested in how auditory and visual speech signals are integrated, and the recent work has revived interest in the role of speech production with respect to this process. Here, we discuss these issues from a developmental perspective. Because speech perception abilities typically outstrip speech production abilities in infancy and childhood, it is unclear how...