The algorithm described in this article is based on the OBS algorithm by Hassibi, Stork and Woll ((1] and 2]). The main disadvantage of OBS is its high complexity. OBS needs to calculate the inverse Hessian to delete only one weight (thus needing much time to prune a big net). A better algorithm should use this matrix to remove more than only one weight, because calculating the inverse Hessian ...

Quasi-Newton algorithms for unconstrained nonlinear minimization generate a sequence of matrices that can be considered as approximations of the objective function second derivatives. This paper gives conditions under which these approximations can be proved to converge globally to the true Hessian matrix, in the case where the Symmetric Rank One update formula is used. The rate of convergence ...

Our goal is to improve variance reducing stochastic methods through better control variates. We first propose a modification of SVRG which uses the Hessian to track gradients over time, rather than to recondition, increasing the correlation of the control variates and leading to faster theoretical convergence close to the optimum. We then propose accurate and computationally efficient approxima...

This paper proposes a modified BFGS formula using a trust region model for solving nonsmooth convex minimizations by using the Moreau-Yosida regularization (smoothing) approach and a new secant equation with a BFGS update formula. Our algorithm uses the function value information and gradient value information to compute the Hessian. The Hessian matrix is updated by the BFGS formula rather than...

Mammography is the most efficient method for breast cancer early detection. Clusters of microcalcifications are the sign of breast cancer and their early detection is the key to improve breast cancer prognosis. Microcalcifications appear in mammogram as tiny granular points, which are difficult to observe by radiologists due to their small size. An efficient method for automatic and accurate de...

The coronary angiography image is easy to be affected by many factors, such as vascular thickness varied huge, complex background noise, uneven illumination intensity and so on. The coronary angiography image is more difficult to deal with compared with other similar medical images. By using Hessian matrix multi-scale vascular detection method, the vicinity of blood vessels will yield a lot of ...

In this paper, a convergent method based on Generalized Iterative Scaling (GIS) with staggered Aitken acceleration is proposed to estimate the parameters for an on-line Conditional Random Field (CRF). The staggered Aitken acceleration method, which alternates between the acceleration and non-acceleration steps, ensures computational simplicity when analyzing incomplete data. The proposed method...

Multidimensional scaling (MDS) is a collection of data analytic techniques for constructing configurations of points from dissimilarity information about interpoint distances. Classsical MDS assumes a fixed matrix of dissimilarities. However, in some applications, e.g., the problem of inferring 3-dimensional molecular structure from bounds on interatomic distances, the dissimilarities are free ...

We propose a very simple, and well principled way of computing the optimal step size in gradient descent algorithms. The on-line version is very efficient computationally, and is applicable to large backpropagation networks trained on large data sets. The main ingredient is a technique for estimating the principal eigenvalue(s) and eigenvector(s) of the objective function's second derivative ma...

We present the gradient and Hessian of the trace of the multivariate Cramér-Rao bound (CRB) formula for unknown impinging angles of plane waves with non-unitary beamspace measurements,. These gradient and Hessian can be used to find the optimal beamspace transformation matrix, i.e., the optimum beamsteering angles, using the Newton-Raphson iteration. These trace formulas are particularly useful...