Audiovisual Synchronization and Fusion Using Canonical Correlation Analysis

Audio-Visual Synchronization and Fusion using Canonical Correlation Analysis

It is well-known that early integration (also called data fusion) is effective when the modalities are correlated, and late integration (also called decision or opinion fusion) is optimal when modalities are uncorrelated. In this paper, we propose a new multimodal fusion strategy for open-set speaker identification using a combination of early and late integration following canonical correlatio...

Generalized canonical correlation analysis for disparate data fusion

Manifold matching works to identify embeddings of multiple disparate data spaces into the same low-dimensional space, where joint inference can be pursued. It is an enabling methodology for fusion and inference from multiple and massive disparate data sources. In this paper we focus on a method called Canonical Correlation Analysis (CCA) and its generalization Generalized Canonical Correlation ...

Canonical correlation analysis using within-class coupling

0167-8655/$ see front matter 2010 Elsevier B.V. A doi:10.1016/j.patrec.2010.09.025 q The work of O. Kursun was supported by Scienti nation Unit of Istanbul University under the grant YA ⇑ Corresponding author. Tel.: +90 212 473 7070/17 E-mail addresses: [email protected] (O. Kurs Alpaydin), [email protected] (O.V. Favorov). Fisher’s linear discriminant analysis (LDA) is one of the most ...

Speech Pattern Discovery using Audio-Visual Fusion and Canonical Correlation Analysis

In this paper, we address the problem of automatic discovery of speech patterns using audio-visual information fusion. Unlike those previous studies based on single audio modality, our work not only uses the acoustic information, but also takes into account the visual features extracted from the mouth region. To improve the effectiveness of the use of multimodal information, several audio-visua...

Predicting Atmospheric Parameters using Canonical Correlation Analysis