A systematic approach to Lyapunov analyses of continuous-time models in convex optimization

05/25/2022
by   Céline Moucer, et al.
0

First-order methods are often analyzed via their continuous-time models, where their worst-case convergence properties are usually approached via Lyapunov functions. In this work, we provide a systematic and principled approach to find and verify Lyapunov functions for classes of ordinary and stochastic differential equations. More precisely, we extend the performance estimation framework, originally proposed by Drori and Teboulle [10], to continuous-time models. We retrieve convergence results comparable to those of discrete methods using fewer assumptions and convexity inequalities, and provide new results for stochastic accelerated gradient flows.

READ FULL TEXT
research
12/27/2021

Last-Iterate Convergence of Saddle Point Optimizers via High-Resolution Differential Equations

Several widely-used first-order saddle point optimization methods yield ...
research
10/05/2018

Continuous-time Models for Stochastic Optimization Algorithms

We propose a new continuous-time formulation for first-order stochastic ...
research
03/14/2016

A Variational Perspective on Accelerated Methods in Optimization

Accelerated gradient methods play a central role in optimization, achiev...
research
12/24/2014

Concentration for matrix martingales in continuous time and microscopic activity of social networks

This paper gives new concentration inequalities for the spectral norm of...
research
03/09/2023

Efficient and Parallel Solution of High-order Continuous Time Galerkin for Dissipative and Wave Propagation Problems

We propose efficient and parallel algorithms for the implementation of t...
research
02/03/2019

Stochastic first-order methods: non-asymptotic and computer-aided analyses via potential functions

We provide a novel computer-assisted technique for systematically analyz...
research
08/02/2020

On unified framework for continuous-time grey models: an integral matching perspective

Since most of the research about grey forecasting models is focused on d...

Please sign up or login with your details

Forgot password? Click here to reset