Using PHiPAC to speed error back-propagation learning

Signal processing algorithms such as neural network learning, convolution, cross-correlation, IIR ltering, etc., can be computationally time-consuming and are often used in time-critical application. This makes it desirable to achieve high e ciency on these routines. Such algorithms are often coded in assembly language to achieve optimal speed, but it is then di cult to make a full exploration of a routine's design space, and the resulting code might be unusable or sub-optimal on di erent systems. Alternatively, the algorithms could be written in a highlevel language and fed to an optimizing compiler. While there is a large literature on relevant compiler techniques [12, 9, 10, 1, 6, 11] that can be used to generate reasonably good code in general, they tend not to generate near-peak code for any one operation. A high-level language's semantics might also obstruct aggressive compiler optimizations. Moreover, it takes signi cant time and investment before compiler research appears in production compilers, so these capabilities are often simply unavailable. We have developed a methodology, named PHiPAC, for developing Portable High-Performance numerical libraries in ANSI C. Our goal is to produce, with minimal e ort, high-performance numerical libraries for a wide range of systems. We brie y describe the methodology in Section 2. Using this methodology, we have produced a portable, BLAScompatible [7], matrix multiply generator. The resulting code can achieve over 90% of peak performance on a variety of current workstations, and is often faster than the vendor-supplied optimized libraries. We use the resulting matrix multiply code to implement a bunch-mode back-propagation training program that uses multiple rather than one training pattern for each weight update. We investigate the tradeo s between bunch size, convergence rate, and training speed using a speech recognition task in Section 3.