New class of limited-memory variationally-derived variable metric methods
نویسندگان
چکیده
A new family of limited-memory variationally-derived variable metric or quasi-Newton methods for unconstrained minimization is given. The methods have quadratic termination property and use updates, invariant under linear transformations. Some encouraging numerical experience is reported.
منابع مشابه
New class of limited - memory variationally - derived variable metric methods 1
We present a new family of limited-memory variationally-derived variable metric (VM) line search methods with quadratic termination property for unconstrained minimization. Starting with x0 ∈ RN , VM line search methods (see [6], [3]) generate iterations xk+1 ∈ RN by the process xk+1 = xk + sk, sk = tkdk, where the direction vectors dk ∈ RN are descent, i.e. g k dk < 0, k ≥ 0, and the stepsizes...
متن کاملGeneralizations of the limited-memory BFGS method based on the quasi-product form of update
Two families of limited-memory variable metric or quasi-Newton methods for unconstrained minimization based on quasi-product form of update are derived. As for the first family, four variants how to utilize the Strang recurrences for the Broyden class of variable metric updates are investigated; three of them use the same number of stored vectors as the limitedmemory BFGS method. Moreover, one ...
متن کاملRecursive formulation of limited memory variable metric methods
In this report we propose a new recursive matrix formulation of limited memory variable metric methods. This approach enables to approximate of both the Hessian matrix and its inverse and can be used for an arbitrary update from the Broyden class (and some other updates). The new recursive formulation requires approximately 4mn multiplications and additions for the direction determination, so i...
متن کاملLimited-memory projective variable metric methods for unconstrained minimization
A new family of limited-memory variable metric or quasi-Newton methods for unconstrained minimization is given. The methods are based on a positive definite inverse Hessian approximation in the form of the sum of identity matrix and two low rank matrices, obtained by the standard scaled Broyden class update. To reduce the rank of matrices, various projections are used. Numerical experience is e...
متن کاملVariable Metric Stochastic Approximation Theory
We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results for learning from expert advice.
متن کامل