Optimization in Deep Learning is mainly guided by vague intuitions and strong assumptions, with a limited understanding how why these work practice. To shed more light on this, our provides some deeper understandings of SGD behaves empirically analyzing the trajectory taken from line search perspective. Specifically, costly quantitative analysis full-batch loss along trajectories common used mo...

Criteria containing a barrier function i.e., an unbounded function at the boundary of the feasible solution domain are frequently encountered in the optimization framework, in particular in interior point methods for constrained optimization. Barrier function has to be carefully handled in the optimization algorithm. When an iterative descent method is used for the minimization, a search along ...

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 13 May 2020Accepted: 02 March 2021Published online: 20 2021Keywordslow-rank matrices, optimization on manifolds, multilevel optimization, Riemannian retraction-based line search, roundoff errorAMS Subject Headings65F10, 65N22, 65F50, 65K10Publication DataISSN (print): 106...

In this paper we present a subgradient method with non-monotone line search for the minimization of convex functions simple constraints. Different from standard prefixed step sizes, new selects sizes in an adaptive way. Under mild conditions asymptotic convergence results and iteration-complexity bounds are obtained. Preliminary numerical illustrate relative efficiency proposed method.

The question we investigate is how to obtain PLC applications with confidence in their proper functioning. Especially, we are interested in the contribution that formal methods can provide for their development. Our maxim is that the place of a particular formal method in the total picture of system development should be made very clear. Developers and customers ought to understand very well wh...