Accelerating incremental gradient optimization with curvature information
نویسندگان
چکیده
منابع مشابه
Accelerating Stochastic Gradient Descent
There is widespread sentiment that fast gradient methods (e.g. Nesterov’s acceleration, conjugate gradient, heavy ball) are not effective for the purposes of stochastic optimization due to their instability and error accumulation. Numerous works have attempted to quantify these instabilities in the face of either statistical or non-statistical errors (Paige, 1971; Proakis, 1974; Polyak, 1987; G...
متن کاملAccelerating Natural Gradient with Higher-Order Invariance
An appealing property of the natural gradient is that it is invariant to arbitrary differentiable reparameterizations of the model. However, this invariance property requires infinitesimal steps and is lost in practical implementations with small but finite step sizes. In this paper, we study invariance properties from a combined perspective of Riemannian geometry and numerical differential equ...
متن کاملExplicit gradient information in multiobjective optimization
This paper presents an algorithm that converges to points that satisfy a first order necessary condition of weakly Pareto solutions of multiobjective optimization problems. Hints on how to include second order information are given. Preliminary numerical results are encouraging.
متن کاملRobust optimization with incremental recourse
In this paper, we consider an adaptive approach to address optimization problems with uncertain cost parameters. Here, the decision maker selects an initial decision, observes the realization of the uncertain cost parameters, and then is permitted to modify the initial decision. We treat the uncertainty using the framework of robust optimization in which uncertain parameters lie within a given ...
متن کاملAccelerating incremental checkpointing for extreme-scale computing
Concern is beginning to grow in the high-performance computing (HPC) community regarding the reliability of future large-scale systems. Disk-based coordinated checkpoint/restart has been the dominant fault tolerance mechanism in HPC systems for the last 30 years. Checkpoint performance is so fundamental to scalability that nearly all capability applications have custom checkpoint strategies to ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Computational Optimization and Applications
سال: 2020
ISSN: 0926-6003,1573-2894
DOI: 10.1007/s10589-020-00183-1