– The main idea of this work is based on the following philosophy: To achieve the ideal compression of audio and video information, utilize the algorithm of information processing intrinsic in the human audio and video systems. In our work, using the results of physiology, musical acoustics and video perception theory, we show that the model of audio and video information processing can be desc...

In this paper, we review recent research that examines audio-visual integration in multimodal communication. The topics include bimodality in human speech, human and automated lip-reading, facial animation, lip synchronization, joint audio-video coding, and bimodal speaker verification. We also study the enabling technologies for these research topics, including automatic facial feature trackin...

In this paper, we present the association of audio and video signatures for short video clip detection. First, we present an audio signature based on the spectral flatness measure. Then we describe a spatio−temporal video signature, based on the evolution of gray level centroids over time. The major contribution of this work is the association of these two signatures in a so−called audio−visual...

The use of audio to retrieve and index the associated video is a relatively new approach. In this paper the focus is on MPEG video. For indexing and retrieval one needs to segment the audio stream associated with the video in terms of gender, speech, music and the speaker. This is called “Audio scene” analysis. The paper discusses techniques for such analysis in the MPEG audio compressed domain.

full text, pictures, video/audio pictures, video/audio comments, interviews, technical graphics, raw data 1)

This paper studies content-based video retrieval using the combination of audio and visual features. The visual feature is extracted by an adaptive video indexing technique that places a strong emphasis on accurate characterization of spatio-temporal information within video clips. Audio feature is extracted by a statistical time-frequency analysis method that applies Laplacian mixture models t...