The popular BFGS quasi-Newton minimization algorithm under reasonable conditions converges globally on smooth convex functions. This result was proved by Powell in 1976: we consider its implications for functions that are not smooth. In particular, an analogous convergence result holds for functions, like the Euclidean norm, that are nonsmooth at the minimizer.

We derive an asymptotic Newton algorithm for Quasi Maximum Likelihood estimation of the ICA mixture model, using the ordinary gradient and Hessian. The probabilistic mixture framework can accommodate non-stationary environments and arbitrary source densities. We prove asymptotic stability when the source models match the true sources. An application to EEG segmentation is given.

A wavelet collocation method for nonlinear Hammerstein equations is formulated. A sparsity in the Jacobian matrix is obtained which gives rise to a fast algorithm for nonlinear solvers such as the Newton’s method and the quasi-Newton method. A fast multilevel augmentation method is developed on a transformed nonlinear equation which gives an additional saving in computational time.

In this paper we provide implementable methods for solving nondifferentiable convex optimization problems. A typical method minimizes an approximate Moreau–Yosida regularization using a quasi-Newton technique with inexact function and gradient values which are generated by a finite inner bundle algorithm. For a BFGS bundle-type method global and superlinear convergence results for the outer ite...

We describe an infeasible interior point algorithm for convex minimization problems. The method uses quasi-Newton techniques for approximating the second derivatives and providing superlinear convergence. We propose a new feasibility control of the iterates by introducing shift variables and by penalizing them in the barrier problem. We prove global convergence under standard conditions on the ...

— In this paper, the performances of the quasi-Newton BFGS algorithm, the NEWUOA derivative free optimizer, the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), the Differential Evolution (DE) algorithm and Particle Swarm Optimizers (PSO) are compared experimentally on benchmark functions reflecting important challenges encountered in real-world optimization problems. Dependence of the...

We propose a robust procedure based on the regularized phase-tracking (RPT) technique to demodulate squared-grating deflectograms. The use of squared gratings, already reported, lets us multiplex the information of the deflections in two orthogonal directions in a single image, thus avoiding the necessity of rotating the gratings. The good noise-rejection characteristics of the RPT technique ar...

This paper analyzes local convergence rates of primal-dual interior point methods for general nonlinearly constrained optimization problems. For this purpose, we first discuss modified Newton methods and modified quasi-Newton methods for solving a nonlinear system of equations, and show local and Qquadratic/Q-superlinear convergence of these methods. These methods are characterized by a perturb...

— In this paper, the performances of the quasi-Newton BFGS algorithm, the NEWUOA derivative free optimizer, the Covariance Matrix Adaptation Evolution Strategy (CMA-ES), the Differential Evolution (DE) algorithm and Particle Swarm Optimizers (PSO) are compared experimentally on benchmark functions reflecting important challenges encountered in real-world optimization problems. Dependence of the...