Limited-memory BFGS quasi-Newton methods approximate the Hessian matrix of second derivatives by the sum of a diagonal matrix and a fixed number of rank-one matrices. These methods are particularly effective for large problems in which the approximate Hessian cannot be stored explicitly. It can be shown that the conventional BFGS method accumulates approximate curvature in a sequence of expandi...

An algorithm for solving large-scale nonlinear' programs with linear constraints is presented. The method combines efficient sparse-matrix techniques as in the revised simplex method with stable quasi-Newton methods for handling the nonlinearities. A general-purpose production code (MINOS) is described, along with computational experience on a wide variety of problems.

A new hybrid technique for optimization of a multivariable function is proposed. This method is applied to the problem of complex time Green’s function of multilayer media. This technique combines Particle Swarm search algorithm with the gradient based quasi-Newton method. Superiority of the method is demonstrated by comparing its results with other optimization techniques.

In this paper, we investigate the semismooth Newton and quasi-Newton methods for the minimization problem in the weighted `−regularization of nonlinear inverse problems. We propose the conditions for obtaining the convergence of two methods. The semismooth Newton method is proven to locally converge with superlinear rate and the semismooth quasi-Newton method is proven to locally converge at le...

A parallel algorithm is proposed for a fundamental problem of machine learning, that of mul-ticategory discrimination. The algorithm is based on minimizing an error function associated with a set of highly structured linear inequalities. These inequalities characterize piecewise-linear separation of k sets by the maximum of k aane functions. The error function has a Lipschitz continuous gradien...

This paper deals with numerical Toeplitz matrix approximation. Our approach is based on (i) a projection algorithm which converges globally but slowly; and (ii) the quasi-Newton method which is faster. Hybrid methods that attempt to combine the best features of both methods are then considered.

We consider proximal gradient methods for minimizing a composite function of differentiable and convex function. To accelerate the general methods, we focus on quasi-Newton type based mappings scaled by matrices. Although it is usually difficult to compute mappings, applying memoryless symmetric rank-one (SR1) formula makes this easier. Since (quasi-Newton) matrices must be positive definite, d...

This paper introduces new contrast functions for blind separation of sources with different time-frequency signatures. Two contrast functions based on the Kullback-Leibler and Jensen-Rényi divergences in the time-frequency (T-F) plane are introduced. Two iterative algorithms are proposed for the proposed contrasts optimization and source separation. One algorithm consists of spatial whitening a...

The adjoint Newton algorithm (ANA) is based on the firstand second-order adjoint techniques allowing one to obtain the ‘‘Newton line search direction’’ by integrating a ‘‘tangent linear model’’ backward in time (with negative time steps). Moreover, the ANA provides a new technique to find Newton line search direction without using gradient information. The error present in approximating the Hes...