Normal-learning children (NL) and children with learning disabilities (LD) reported their perceptions of unisensory (auditory or visual), concordant audiovisual (e.g. visual /apa/ and auditory /apa/) and conflicting (e.g. visual /aka/ and auditory /apa/) speech stimuli in quiet and noise (0 dB and -12 dB signal-to-noise ratio, SNR). In normal populations, watching such conflicting combinations ...

This paper presents an algorithm to correlate audio and visual data generated by the same physical phenomenon. According to psychophysical experiments, temporal synchrony strongly contributes to integrate cross-modal information in humans. Thus, we define meaningful audiovisual structures as temporally proximal audio-video events. Audio and video signals are represented as sparse decompositions...

Audiovisual performance is a fertile area for creative expression, an intersection of experimental cinema and computer music that has seen a groundswell of interest in recent years. To create works in this emerging medium, a complex network of relationships between sounds, images and sensor input must be organized. This complexity poses major technical and aesthetic challenges which a systemati...

............................................................................................... v Introduction............................................................................................. 1 Review of the literature ............................................................................... 3 Auditory speech perception ............................................................

Modern interactive means combined with new digital media processing and representation technologies can provide a robust framework for enhancing user experience in multimedia entertainment systems and audiovisual artistic installations with non-traditional interaction/feedback paths based on user affective state. In this work, the ‘Elevator’ interactive audiovisual platform prototype is present...

Many of today’s audiovisual application systems offer some kind of interactivity. Yet, quality assessments of these systems are often performed without taking into account the possible effects of divided attention caused by interaction or user task. We present a subjective assessment performed among 40 test subjects to investigate the impact of divided attention on the perception of audiovisual...

Speech is the most important form of human communication but ambient sounds and competing talkers often degrade its acoustics. Fortunately the brain can use visual information, especially its highly precise spatial information, to improve speech comprehension in noisy environments. Previous studies have demonstrated that audiovisual integration depends strongly on spatiotemporal factors. Howeve...

Although there has been a great deal of recent empirical work and new theoretical interest in audio-visual perception of speech in both normal-hearing and hearing-impaired adults, relatively little is known about the development of these abilities and skills in deaf children with cochlear implants (CIs). This study examined the way in which children integrate visual speech cues available in the...

Many organisms sample their environment through multiple sensory systems and the integration of multisensory information enhances learning. However, the mechanisms underlying multisensory memory formation and their similarity to unisensory mechanisms remain unclear. Filial imprinting is one example in which experience is multisensory, and the mechanisms of unisensory neuronal plasticity are wel...

In a recent study in younger adults (19-29 year olds) we showed evidence that distributed audiovisual attention resulted in improved discrimination performance for audiovisual stimuli compared to focused visual attention. Here, we extend our findings to healthy older adults (60-90 year olds), showing that performance benefits of distributed audiovisual attention in this population match those o...