Gauss mixture vector quantization

Gauss mixtures are a popular class of models in statistics and statistical signal processing because they can provide good £ts to smooth densities, because they have a rich theory, and because the can be well estimated by existing algorithms such as the EM algorithm. We here extend an information theortic extremal property for source coding from Gaussian sources to Gauss mixtures using high rate quantization theory and extend a method originally used for LPC speech vector quantization to provide a Lloyd clustering approach to the design of Gauss mixture models. The theory provides formulas relating minimum discrimination information (MDI) for model selection and the mean squared error resulting when the MDI criterion is used in an optimized robust classi£ed vector quantizer. It also provides motivation for the use of Gauss mixture models for robust compression systems for general random vectors.