Fast Parallel Estimation of High Dimensional Information Theoretical Quantities with Low Dimensional Random Projection Ensembles
نویسندگان
چکیده
Goal: estimation of high dimensional information theoretical quantities (entropy, mutual information, divergence). • Problem: computation/estimation is quite slow. • Consistent estimation is possible by nearest neighbor (NN) methods [1] → pairwise distances of sample points: – expensive in high dimensions [2], – approximate isometric embedding into low dimension is possible (Johnson-Lindenstrauss (JL) Lemma [3], random projection (RP) [4]), – idea: estimation using the embedded low dimensional samples. Demo: estimation of multidimensional differential entropy → Independent Subspace Analysis (ISA) task [5].
منابع مشابه
Distributed high dimensional information theoretical image registration via random projections
Information theoretical measures, such as entropy, mutual information, and various divergences, exhibit robust characteristics in image registration applications. However, the estimation of these quantities is computationally intensive in high dimensions. On the other hand, consistent estimation from pairwise distances of the sample points is possible, which suits random projection (RP) based l...
متن کاملInvestigation of pore-scale random porous media using lattice boltzmann method
The permeability and tortuosity of pore-scale two and three-dimensional random porous media were calculated using the Lattice Boltzmann method (LBM). Effects of geometrical parameters of medium on permeability and tortuosity were investigated as well. Two major models of random porous media were reconstructed by computerized tomography method: Randomly distributed rectangular obstacles in a uni...
متن کاملCluster Ensembles for High Dimensional Clustering: An Empirical Study
This paper studies cluster ensembles for high dimensional data clustering. We examine three different approaches to constructing cluster ensembles. To address high dimensionality, we focus on ensemble construction methods that build on two popular dimension reduction techniques, random projection and principal component analysis (PCA). We present evidence showing that ensembles generated by ran...
متن کاملHigh Dimensional Data Fusion via Joint Manifold Learning
The emergence of low-cost sensing architectures for diverse modalities has made it possible to deploy sensor networks that acquire large amounts of very high-dimensional data. To cope with such a data deluge, manifold models are often developed that provide a powerful theoretical and algorithmic framework for capturing the intrinsic structure of data governed by a low-dimensional set of paramet...
متن کاملLearning in high dimensions with projected linear discriminants
The enormous power of modern computers has made possible the statistical modelling of data with dimensionality that would have made this task inconceivable only decades ago. However, experience in such modelling has made researchers aware of many issues associated with working in high-dimensional domains, collectively known as ‘the curse of dimensionality’, which can confound practitioners’ des...
متن کامل