However, there are strong connections between these approaches when we deal with a binary target attribute. In this particular case, we can even recreate the outputs of the linear discriminant analysis with a linear regression program (Bishop, 2007, pages 189 190; Duda et al., 2001, pages 242 – 243; Huberty et Olejnik, 2006, pages 353 – 355; Nakache et Confais, 2003, pages 14 – 16; Saporta, 200...

Adaptative training of the non-linear single-layer perceptron can lead to the Euclidean distance classifier and later to the standard Fisher linear discriminant function. On the way between these two classifiers one has a regularized discriminant analysis. That is equivalent to the “weight decay” regularization term added to the cost function. Thus early stopping plays a role of regularization ...