Geometric Dependency Analysis

We investigate methods for estimating and visualising the residual dependency structure in distributed representations such as those produced by independent component analysis (ICA). This process involves estimating the mutual information between many pairs of components, for which use a method based on nonlinear correlations. The pairwise dependencies are then represented geometrically using multidimensional scaling (MDS) in such a way that strongly dependent components are brought close together. The method is applied to ICA representations of speech and music audio signals. The resulting geometric structures reflect the time-frequency structure of the speech-derived ICA representation and a more complex relationship, related to the harmonic structure of Western musical scales, between frequencies in the music-derived representation.