Extrapolation methods use the last few iterates of an optimization algorithm to produce a better estimate of the optimum. They were shown to achieve optimal convergence rates in a deterministic setting using simple gradient iterates. Here, we study extrapolation methods in a stochastic setting, where the iterates are produced by either a simple or an accelerated stochastic gradient algorithm. W...

We study local optima of the Hamiltonian of the Sherrington-Kirkpatrick model. We compute the exponent of the expected number of local optima and determine the “typical” value of the Hamiltonian. *Supported by NSERC Discovery Grant 341845. †Supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada. ‡Supported by the SpanishMinistry of Economy and Competitiveness, Gran...