Analysis of Kinetic Models for Label Switching and Stochastic Gradient Descent

07/01/2022
by   Martin Burger, et al.
0

In this paper we provide a novel approach to the analysis of kinetic models for label switching, which are used for particle systems that can randomly switch between gradient flows in different energy landscapes. Besides problems in biology and physics, we also demonstrate that stochastic gradient descent, the most popular technique in machine learning, can be understood in this setting, when considering a time-continuous variant. Our analysis is focusing on the case of evolution in a collection of external potentials, for which we provide analytical and numerical results about the evolution as well as the stationary problem.

READ FULL TEXT
research
10/20/2022

A note on diffusion limits for stochastic gradient descent

In the machine learning literature stochastic gradient descent has recen...
research
02/02/2021

Stability and Generalization of the Decentralized Stochastic Gradient Descent

The stability and generalization of stochastic gradient-based methods pr...
research
02/14/2022

Continuous-time stochastic gradient descent for optimizing over the stationary distribution of stochastic differential equations

We develop a new continuous-time stochastic gradient descent method for ...
research
05/22/2018

Efficient Stochastic Gradient Descent for Distributionally Robust Learning

We consider a new stochastic gradient descent algorithm for efficiently ...
research
12/03/2020

SSGD: A safe and efficient method of gradient descent

With the vigorous development of artificial intelligence technology, var...
research
09/28/2018

Fluctuation-dissipation relations for stochastic gradient descent

The notion of the stationary equilibrium ensemble has played a central r...
research
08/24/2020

Stochastic Gradient Descent Works Really Well for Stress Minimization

Stress minimization is among the best studied force-directed graph layou...

Please sign up or login with your details

Forgot password? Click here to reset