To efficiently solve a large scale unconstrained minimization problem with a dense Hessian matrix, this paper proposes to use an incomplete Hessian matrix to define a new modified Newton method, called the incomplete Hessian Newton method (IHN). A theoretical analysis shows that IHN is convergent globally, and has a linear rate of convergence with a properly selected symmetric, positive definit...

Stable techniques are considered for updating the reduced Hessian matrix that arises in a null{space active set method for Quadratic Programming when the Hessian matrix itself may be indeenite. A scheme for deening and updating the null-space basis matrix is described which is adequately stable and allows advantage to be taken of sparsity. A new canonical form for the reduced Hessian matrix is ...

Limited-memory BFGS quasi-Newton methods approximate the Hessian matrix of second derivatives by the sum of a diagonal matrix and a fixed number of rank-one matrices. These methods are particularly effective for large problems in which the approximate Hessian cannot be stored explicitly. It can be shown that the conventional BFGS method accumulates approximate curvature in a sequence of expandi...

Previously, Bottou and LeCun [1] established that the second-order stochastic gradient descent (SGD) method can potentially achieve generalization performance as well as empirical optimum in a single pass through the training examples. However, second-order SGD requires computing the inverse of the Hessian matrix of the loss function, which is usually prohibitively expensive. Recently, we inven...

For functions defined on integer lattice points, discrete versions of the Hessian matrix have been considered in various contexts. In discrete convex analysis, for example, certain combinatorial properties of the discrete Hessian matrices are known to characterize M-convex and L-convex functions, which can be extended to convex functions in real variables. The relationship between convex extens...

this paper presents a new procedure for automatic extraction of the blood vessels and optic disk (od) in fundus fluorescein angiogram (ffa). in order to extract blood vessel centerlines, the algorithm of vessel extraction starts with the analysis of directional images resulting from sub‑bands of fast discrete curvelet transform (fdct) in the similar directions and different scales. for this pur...

We propose a modification to the simultaneous perturbation stochastic approximation (SPSA) methods based on the comparisons made between the firstand second-order SPSA (1SPSA and 2SPSA) algorithms from the perspective of loss function Hessian. At finite iterations, the accuracy of the algorithm depends on the matrix conditioning of the loss function Hessian. The error of 2SPSA algorithm for a l...

The score and hessian functions The score vector ∇L(β, x) and hessian matrix ∇ 2 L(β, x) used in the Newton-Raphson

In this paper we propose a novel approach to measuring curvature in color or vector-valued images (up to 4-dimensions) based on quaternion singular value decomposition of a Hessian matrix. This approach generalizes the existing scalar-image curvature approach which makes use of the eigenvalues of the Hessian matrix [1]. In the case of vector-valued images, the Hessian is no longer a 2D matrix b...

The Covariance Matrix Adaptation Evolution Strategy (CMAES) has been the most successful Evolution Strategy at exploiting covariance information; it uses a form of Principle Component Analysis which, under certain conditions, is suggested to converge to the correct covariance matrix, formulated as the inverse of the mathematically well-defined Hessian matrix. However, in practice, there exist c...