On the Koopman operator of algorithms

07/25/2019
by   Felix Dietrich, et al.
0

A systematic mathematical framework for the study of numerical algorithms would allow comparisons, facilitate conjugacy arguments, as well as enable the discovery of improved, accelerated, data-driven algorithms. Over the course of the last century, the Koopman operator has provided a mathematical framework for the study of dynamical systems, which facilitates conjugacy arguments and can provide efficient reduced descriptions. More recently, numerical approximations of the operator have made it possible to analyze dynamical systems in a completely data-driven, essentially equation-free pipeline. Discrete or continuous time numerical algorithms (integrators, nonlinear equation solvers, optimization algorithms) are themselves dynamical systems. In this paper, we use the Koopman operator framework in the data-driven study of such algorithms and discuss benefits for analysis and acceleration of numerical computation. For algorithms acting on high-dimensional spaces by quickly contracting them towards low-dimensional manifolds, we demonstrate how basis functions adapted to the data help to construct efficient reduced representations of the operator. Our illustrative examples include the gradient descent and Nesterov optimization algorithms, as well as the Newton-Raphson algorithm.

READ FULL TEXT
research
01/13/2022

Data-Driven Modeling and Prediction of Non-Linearizable Dynamics via Spectral Submanifolds

We develop a methodology to construct low-dimensional predictive models ...
research
12/06/2019

Optimization algorithms inspired by the geometry of dissipative systems

Accelerated gradient methods are a powerful optimization tool in machine...
research
02/10/2023

On the approximability of Koopman-based operator Lyapunov equations

Lyapunov functions play a vital role in the context of control theory fo...
research
06/22/2020

Predicting Rare Events in Multiscale Dynamical Systems using Machine Learning

We study the problem of rare event prediction for a class of slow-fast n...
research
08/21/2019

Data-driven model reduction, Wiener projections, and the Mori-Zwanzig formalism

First-principles models of complex dynamic phenomena often have many deg...
research
09/14/2022

Algorithmic (Semi-)Conjugacy via Koopman Operator Theory

Iterative algorithms are of utmost importance in decision and control. W...
research
04/29/2019

New optimization algorithms for neural network training using operator splitting techniques

In the following paper we present a new type of optimization algorithms ...

Please sign up or login with your details

Forgot password? Click here to reset