A Direction Set Based Algorithm for Adaptive Least Squares Problems in Signal Processing

A fast algorithm, called the direction set based algorithm, has been developed recently for solving a class of adaptive least squares problems arising in signal processing. The algorithm is based on the direction set method developed by Powell and Zangwill for solving unconstrained minimization problems without using derivatives. It is designed so as to fully take advantage of the special structure of the adaptive least squares problems. It is a fast algorithm because it requires only O(N) multiplications for each system update where N is the number of parameters used in the system design. The algorithm has been implemented and applied to applications in signal processing. Computer simulation results have shown that the algorithm is stable and converges fast often with a rate which is comparable to that of the known Conjugate Gradient algorithm and Recursive Least Squares algorithm. The algorithm has been proved to converge linearly for adaptive least squares problems in general and its rate of convergence can be faster than linear for some applications. The algorithm can be simpli ed and its rate of convergence can be improved with some direction sets. The direction set based algorithm has also been modi ed to solve constrained adaptive least squares problems arising in spectral estimation. Computer simulations illustrate that the algorithm is e ective for spectral estimation.