Optimization algorithms inspired by the geometry of dissipative systems

12/06/2019
by   Alessandro Bravetti, et al.
0

Accelerated gradient methods are a powerful optimization tool in machine learning and statistics but their development has traditionally been driven by heuristic motivations. Recent research, however, has demonstrated that these methods can be derived as discretizations of dynamical systems, which in turn has provided a basis for more systematic investigations, especially into the structure of those dynamical systems and their structure preserving discretizations. In this work we introduce dynamical systems defined through a contact geometry which are not only naturally suited to the optimization goal but also subsume all previous methods based on geometric dynamical systems. These contact dynamical systems also admit a natural, robust discretization through geometric contact integrators. We demonstrate these features in paradigmatic examples which show that we can indeed obtain optimization algorithms that achieve oracle lower bounds on convergence rates while also improving on previous proposals in terms of stability.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/25/2019

On the Koopman operator of algorithms

A systematic mathematical framework for the study of numerical algorithm...
research
12/02/2021

Breaking the Convergence Barrier: Optimization via Fixed-Time Convergent Flows

Accelerated gradient methods are the cornerstones of large-scale, data-d...
research
09/05/2019

Numerical integration in celestial mechanics: a case for contact geometry

Several dynamical systems of interest in celestial mechanics can be writ...
research
02/10/2018

On Symplectic Optimization

Accelerated gradient methods have had significant impact in machine lear...
research
03/11/2019

Conformal Symplectic and Relativistic Optimization

Although momentum-based optimization methods have had a remarkable impac...
research
12/03/2022

Distributionally Robust Lyapunov Function Search Under Uncertainty

This paper develops methods for proving Lyapunov stability of dynamical ...
research
11/28/2012

Nature-Inspired Mateheuristic Algorithms: Success and New Challenges

Despite the increasing popularity of metaheuristics, many crucially impo...

Please sign up or login with your details

Forgot password? Click here to reset