Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization

02/23/2021
by   Peiyuan Zhang, et al.
0

Viewing optimization methods as numerical integrators for ordinary differential equations (ODEs) provides a thought-provoking modern framework for studying accelerated first-order optimizers. In this literature, acceleration is often supposed to be linked to the quality of the integrator (accuracy, energy preservation, symplecticity). In this work, we propose a novel ordinary differential equation that questions this connection: both the explicit and the semi-implicit (a.k.a symplectic) Euler discretizations on this ODE lead to an accelerated algorithm for convex programming. Although semi-implicit methods are well-known in numerical analysis to enjoy many desirable features for the integration of physical systems, our findings show that these properties do not necessarily relate to acceleration.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/11/2019

Acceleration via Symplectic Discretization of High-Resolution Differential Equations

We study first-order optimization methods obtained by discretizing ordin...
research
05/17/2019

A Dynamical Systems Perspective on Nesterov Acceleration

We present a dynamical system framework for understanding Nesterov's acc...
research
09/20/2017

Survey on Semi-Explicit Time Integration of Eddy Current Problems

The spatial discretization of the magnetic vector potential formulation ...
research
06/21/2023

Stability analysis of an implicit and explicit numerical method for Volterra integro-differential equations with kernel K(x,y(t),t)

We present implicit and explicit versions of a numerical algorithm for s...
research
05/18/2021

A Contraction Theory Approach to Optimization Algorithms from Acceleration Flows

Much recent interest has focused on the design of optimization algorithm...
research
07/17/2022

Parallelizing Explicit and Implicit Extrapolation Methods for Ordinary Differential Equations

Numerically solving ordinary differential equations (ODEs) is a naturall...
research
05/27/2022

ARKODE: a flexible IVP solver infrastructure for one-step methods

We describe the ARKODE library of one-step time integration methods for ...

Please sign up or login with your details

Forgot password? Click here to reset