Phase transitions in vector quantization and neural gas

The statistical physics of off-learning is applied to Winner-Takes-All and rank-based vector quantization (VQ), including the Neural Gas (NG). The analysis is based on the limit of high training temperatures and the annealed approximation. The typical learning behavior is evaluated for systems of two and three prototypes with data drawn from a mixture of high dimensional Gaussian clusters. The learning curves exhibit phase transitions, i.e. a critical or discontinuous dependence of performances on the training set size and training temperature. We show how the nature and properties of the transition depend on the number of prototypes and the control parameter of rank based cost functions. The NG based systems are demonstrated to give an advantage over WTA in terms of robustness to initial conditions.