A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights

We derive a second-order ordinary differential equation (ODE), which is the limitof Nesterov’s accelerated gradient method. This ODE exhibits approximate equiv-alence to Nesterov’s scheme and thus can serve as a tool for analysis. We show thatthe continuous time ODE allows for a better understanding of Nesterov’s scheme.As a byproduct, we obtain a family of schemes with similar convergence rates.The ODE interpretation also suggests restarting Nesterov’s scheme leading to analgorithm, which can be rigorously proven to converge at a linear rate wheneverthe objective is strongly convex.