Independent Component Analysis by Minimization

Independent component analysis (ICA) is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. In this paper, the linear version of the ICA problem is approached from an information-theoretic viewpoint, using Comon's framework of minimizing mutual information of the components. Using maximum entropy approximations of diierential entropy, we introduce a family of new contrast (objective) functions for ICA, which can also be considered 1-D projection pursuit indexes. The statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model. It is shown how to choose optimal contrast functions according to diierent criteria. Novel algorithms for maximizing the contrast functions are then introduced. Hebbian-like learning rules are shown to result from gradient descent methods. Finally, in order to speed up the convergence, a family of xed-point algorithms for maximization of the contrast functions is introduced.