This paper presents an audio-visual data representation for violent scenes detection in Hollywood movies. Existing works in this field consider either the audio or the visual information; or their shallow fusion. None has yet explored their joint dependence for violent scenes detection. We propose a feature which provides strong multimodal audio and visual cues by first joining the audio and th...

Humans are often able to compensate for noise degradation and uncertainty in speech information by augmenting the received audio with visual information. Such bimodal perception generates a rich combination of information that can be used in the recognition of speech. However, due to wide variability in the lip movement involved in articulation, not all speech can be substantially improved by a...

Deaf people cannot properly access the speech information stored in any kind of recording format (audio, video, etc). We present a system that provides with subtitling and Spanish Sign Language representation capabilities to allow Spanish Deaf population can access to such speech content. The system is composed by a speech recognition module, a machine translation module from Spanish to Spanish...

This article applies Boutet’s kinesiological approach (2018) to a study of the expression negation in four children evolving contrastive language environments: predominantly monolingual (English, French, LSF) or bilingual (LSF-French), unimodal (visual modality- gestural) bimodal (visuo-gestural and audio-vocal). We its developmental multimodal aspects according an integrated model that views g...

The use of visual features in audio-visual speech recognition (AVSR) is justified by both the speech generation mechanism, which is essentially bimodal in audio and visual representation, and by the need for features that are invariant to acoustic noise perturbation. As a result, current AVSR systems demonstrate significant accuracy improvements in environments affected by acoustic noise. In th...

Background. A field study conducted by researchers found the balls used by visual impairment students at school basically used the sound by a small bell inside the ball. However, the sound emitted from the ball is very limited, the ball will sound when it is moved. This makes it difficult for students with visual impairments to find a missing ball that not emitted makes a sound. Objectives. A ...

The use of visual information in addition to acoustic can improve automatic speech recognition. In this paper we compare different approaches for audio-visual information integration and show how they affect automatic speech recognition performance. We utilize Facial Animation Parameters (FAPs), supported by the MPEG-4 standard for the visual representation as visual features. We use both Singl...

Due to the fact that most offices and educational institutions now operate from home, work-from-home study-from-home cultures have made it difficult interact with persons who are deaf or hard of hearing. These people communicate within their society using sign language, which is not widely understood by others. Most time, as a result this, they miss out on opportunity express point in front eve...