Adaptability of the backpropagation procedure

Possible paradigms for concept learning by feedfor-ward neural networks include discrimination and recognition. An interesting aspect of this di-chotomy is that the recognition-based implementation can learn certain domains much more ee-ciently than the discrimination-based one, despite the close structural relationship between the two systems. The purpose of this paper is to explain this diierence in eeciency. We suggest that it is caused by a diierence in the generalization strategy adopted by the Backpropagation procedure in both cases: while the autoassociator uses a (fast) bottom-up strategy, MLP has recourse to a (slow) top-down one, despite the fact that the two systems are both optimized by the Backpropagation procedure. This result is important because it sheds some light on the nature of Backpropagation's adaptative capability. From a practical viewpoint, it suggests a deterministic way to increase the eeciency of Backpropagation-trained feedforward networks.