An Improved Tikhonov-Regularized Variable Projection Algorithm for Separable Nonlinear Least Squares

Variable projection for nonlinear least squares problems

The variable projection algorithm of Golub and Pereyra (1973) has proven to be quite valuable in the solution of nonlinear least squares problems in which a substantial number of the parameters are linear. Its advantages are efficiency and, more importantly, a better likelihood of finding a global minimizer rather than a local one. The purpose of our work is to provide a more robust implementat...

An Efficient Algorithm for the Separable Nonlinear Least Squares Problem

The nonlinear least squares problem miny,z‖A(y)z + b(y)‖, where A(y) is a full-rank (N + `)× N matrix, y ∈ Rn, z ∈ RN and b(y) ∈ RN+` with ` ≥ n, can be solved by first solving a reduced problem miny‖ f (y)‖ to find the optimal value y∗ of y, and then solving the resulting linear least squares problem minz‖A(y∗)z + b(y∗)‖ to find the optimal value z∗ of z. We have previously justified the use o...

Separable Nonlinear Least Squares: the Variable Projection Method and its Applications

Optimal Rates for Regularized Least-squares Algorithm

We develop a theoretical analysis of the generalization performances of regularized least-squares algorithm on a reproducing kernel Hilbert space in the supervised learning setting. The presented results hold in the general framework of vector-valued functions, therefore they can be applied to multi-task problems. In particular we observe that the concept of effective dimension plays a central ...

Fast Rates for Regularized Least-squares Algorithm

We develop a theoretical analysis of generalization performances of regularized leastsquares on reproducing kernel Hilbert spaces for supervised learning. We show that the concept of effective dimension of an integral operator plays a central role in the definition of a criterion for the choice of the regularization parameter as a function of the number of samples. In fact a minimax analysis is...