In this paper, we propose a novel sufficient decrease technique for variance reduced stochastic gradient descent methods such as SAG, SVRG and SAGA. In order to make sufficient decrease for stochastic optimization, we design a new sufficient decrease criterion, which yields sufficient decrease versions of variance reduction algorithms such as SVRG-SD and SAGA-SD as a byproduct. We introduce a c...

‎Based on an eigenvalue analysis‎, ‎a new proof for the sufficient‎ ‎descent property of the modified Polak-Ribière-Polyak conjugate‎ ‎gradient method proposed by Yu et al‎. ‎is presented‎.

In this paper, we derive a new parameter µk-1 for the three-term CG (N3T) algorithm solving unconstrained optimization problems. As demonstrated by its calculations and proof, worth is determined T , study mentions four different types of . The search directions are always sufficiently descent when using strong Wolfe line (SWC). Under reasonable assumptions, proposed achieves global convergence...

A characterization of descent morphism in the category of Priestley spaces, as well as necessary and sufficient conditions for such morphisms to be effective are given. For that we embed this category in suitable categories of preordered topological spaces were descent and effective morphisms are described using the monadic description of descent.

An algorithm and associated strategy for solving polynomial systems within the optimization framework is presented. The algorithm and strategy are named, respectively, the penetrating gradient algorithm and the deepest descent strategy. The most prominent feature of penetrating gradient algorithm, after which it was named, is its ability to “see and penetrate through” the obstacles in error spa...