Abstract Optimization problems that include regularization functions in their objectives are regularly solved many applications. When one seeks second-order methods for such problems, it may be desirable to exploit specific properties of some these when accounting curvature information the solution steps speed up convergence. In this paper, we propose SCORE (self-concordant regularization) fram...

Newton’s method is a classical method for solving a nonlinear equation F (z) = 0. We derive inexact Newton steps that lead to an inexact Newton method, applicable near a solution. The method is based on solving for a particular F (zk′) during p consecutive iterations k = k′, k′ + 1, . . . , k′ + p − 1. One such p-cycle requires 2 − 1 solves with the matrix F (zk′). If matrix factorization is us...

where F (ξ) is a nonlinear operator from a real Euclidean space of dimension n or Hilbert space into itself. The Euclidean norm and corresponding inner product will be denoted by ‖·‖1 and (·, ·)1 respectively. A general different inner product with a weight function and the corresponding norm will be denoted by (·, ·)0 and ‖ · ‖ respectively. In the first part of this article (Sects. 2 and 3) w...

The adjoint Newton algorithm (ANA) is based on the firstand second-order adjoint techniques allowing one to obtain the ‘‘Newton line search direction’’ by integrating a ‘‘tangent linear model’’ backward in time (with negative time steps). Moreover, the ANA provides a new technique to find Newton line search direction without using gradient information. The error present in approximating the Hes...

We discuss the solution of numerically ill-posed overdetermined systems of equations using Tikhonov a-priori-based regularization. When the noise distribution on the measured data is available to appropriately weight the fidelity term, and the regularization is assumed to be weighted by inverse covariance information on the model parameters, the underlying cost functional becomes a random varia...

The purpose of this work was to investigate the utility of implicit integration methods for molecular dynamics (MD) simulation. Implicit methods were investigated in an attempt to address the bottleneck associated with short, stability-limited step size lengths available to the explicit methods currently in widespread use. Two implicit methods were compared to the current standard for time inte...

We introduce a new complexity measure of a path of (problems, solutions) pairs in terms of the length of the path in the condition metric which we define in the article. The measure gives an upper bound for the number of Newton steps sufficient to approximate the path discretely starting from one end and thus produce an approximate zero for the endpoint. This motivates the study of short paths ...

We consider a problem of simultaneous reduction of a sequence of matrices by means of orthogonal transformations. We show that such reduction can be performed by a series of the deflation steps. At each deflation step a simultaneous eigenvalue problem, which is a direct generalization of the generalized eigenvalue problem, is solved. A fast variant of Gauss-Newton algorithm for its solution was...

We study constrained and unconstrained optimization programs for nonconvex maximum eigenvalue functions. We show how second order techniques may be introduced as soon as it is possible to reliably guess the multiplicity of the maximum eigenvalue at a limit point. We examine in which way standard and projected Newton steps may be combined with a nonsmooth first-order method to obtain a globally ...

In this work, we consider block-Jacobi methods with Newton steps in each subspace search and prove their local quadratic convergence to a local minimum with non-degenerate Hessian under some orthogonality assumptions on the search directions. Moreover, such a method is exemplified for non-unitary joint matrix diagonalization, where we present a block-Jacobi-type method on the oblique manifold w...