Compressive Classification of a Mixture of Gaussians: Analysis, Designs and Geometrical Interpretation

This paper derives fundamental limits on the performance of compressive classification when the source is a mixture of Gaussians. It provides an asymptotic analysis of a Bhattacharya based upper bound on the misclassification probability for the optimal Maximum-A-Posteriori (MAP) classifier that depends on quantities that are dual to the concepts of diversity-order and coding gain in multi-antenna communications. The diversity-order of the measurement system determines the rate at which the probability of misclassification decays with signal-to-noise ratio (SNR) in the low-noise regime. The counterpart of coding gain is the measurement gain which determines the power offset of the probability of misclassification in the low-noise regime. These two quantities make it possible to quantify differences in misclassification probability between random measurement and (diversity-order) optimized measurement. Results are presented for two-class classification problems first with zero-mean Gaussians then with nonzero-mean Gaussians, and finally for multiple-class Gaussian classification problems. The behavior of misclassification probability is revealed to be intimately related to certain fundamental geometric quantities determined by the measurement system, the source and their interplay. Numerical results, representative of compressive classification of a mixture of Gaussians, demonstrate alignment of the actual misclassification probability with the Bhattacharya based upper bound. The connection between the misclassification performance and the alignment between source and measurement geometry may be used to guide the design of dictionaries for compressive classification. Index Terms Compressed sensing, compressive classification, reconstruction, classification, random projections, measurement design, dimensionality reduction, Gaussian mixture models, phase transitions, diversity gain, measurement gain. This paper was presented in part at the 2013 IEEE International Symposium on Information Theory and the 2013 IEEE Global Conference on Signal and Information Processing. The work of H. Reboredo was supported by Fundação para a Ciência e Tecnologia, Portugal, through the doctoral grant SFRH/BD/81543/2011. The work of M. R. D. Rodrigues was supported by the EPSRC through the research grant EP/K503459/1. This work was also supported by the Royal Society International Exchanges Scheme IE120996. H. Reboredo and F. Renna are with the Instituto de Telecomunicações and the Departamento de Ciência de Computadores da Faculdade de Ciências da Universidade do Porto, Portugal (email: {hugoreboredo, frarenna}@dcc.fc.up.pt). R. Calderbank is with the Department of Electrical and Computer Engineering, Duke University, NC, USA (email: [email protected]). M. R. D. Rodrigues is with the Department of Electronic and Electrical Engineering, University College London, United Kingdom (email: [email protected]). January 28, 2014 DRAFT 2