Mixtures of Principal Component Analyzers

Principal component analysis (PCA) is a ubiquitous technique for data analysis but one whose effective application is restricted by its global linear character. While global nonlinear variants of PCA have been proposed, an alternative paradigm is to capture data nonlinearity by a mixture of local PCA models. However, existing techniques are limited by the absence of a probabilistic formalism with an appropriate likelihood measure and so require an arbitrary choice of implementation strategy. This paper shows how PCA can be derived from a maximum–likelihood procedure, based on a specialisation of factor analysis. This is then extended to develop a well–defined mixture model of principal component analyzers, and an expectation–maximisation algorithm for estimating all the model parameters is given.