Discriminative training of tied-mixture HMM by deterministic annealing

A deterministic annealing algorithm for the design of tiedmixture HMM recognizers is proposed, which reduces the training sensitivity to parameter initialization, automatically smoothes the classification error cost function to allow gradientbased optimization, and seeks better solutions than known techniques. The new approach introduces randomness into the classification rule during the training process, and minimizes the expected error rate while controlling the level of randomness via a constraint on the Shannon entropy. As the entropy constraint is gradually relaxed, the effective cost function converges to the classification error rate and the system becomes a hard (nonrandom) recognizer. Experiments show that the proposed method outperforms design by maximum likelihood reestimation and by generalized probabilistic descent.