The learning dynamics of the back-propagation algorithm are investigated when complexity constraints are added to the standard Least Mean Square (LMS) cost function. It is shown that loss of generalization performance due to overtraining can be avoided when using such complexity constraints. Furthermore, "energy," hidden representations and weight distributions are observed and compared during ...

With repeated exposure to a distorted visual environment, an individual perceives less distortion on each subsequent occasion. This positive advantage of repeated exposure is found to transfer to a different distortion, provided the subject has not had too much experience with the first distortion. Overtraining on one distortion cancels any positive transfer of advantage from one distortion to ...

Discriminative training of hidden Markov models (HMMs) using segmental minimum classi cation error (MCE) training has been shown to work extremely well for certain speech recognition applications. It is, however, somewhat prone to overspecialization. This study investigates various techniques which improve performance and generalization of the MCE algorithm. Improvements of up to 7% in relative...