DeepAI AI Chat

Analysis of Kinetic Models for Label Switching and Stochastic Gradient Descent

07/01/2022

∙

by Martin Burger, et al.

∙

∙

In this paper we provide a novel approach to the analysis of kinetic models for label switching, which are used for particle systems that can randomly switch between gradient flows in different energy landscapes. Besides problems in biology and physics, we also demonstrate that stochastic gradient descent, the most popular technique in machine learning, can be understood in this setting, when considering a time-continuous variant. Our analysis is focusing on the case of evolution in a collection of external potentials, for which we provide analytical and numerical results about the evolution as well as the stationary problem.

Martin Burger
33 publications
Alex Rossi
2 publications

∙ 10/20/2022

A note on diffusion limits for stochastic gradient descent

In the machine learning literature stochastic gradient descent has recen...

0 Alberto Lanconelli, et al. ∙

∙ 02/02/2021

Stability and Generalization of the Decentralized Stochastic Gradient Descent

The stability and generalization of stochastic gradient-based methods pr...

23 Tao Sun, et al. ∙

∙ 02/14/2022

Continuous-time stochastic gradient descent for optimizing over the stationary distribution of stochastic differential equations

We develop a new continuous-time stochastic gradient descent method for ...

0 Ziheng Wang, et al. ∙

∙ 05/22/2018

Efficient Stochastic Gradient Descent for Distributionally Robust Learning

We consider a new stochastic gradient descent algorithm for efficiently ...

0 Soumyadip Ghosh, et al. ∙

∙ 12/03/2020

SSGD: A safe and efficient method of gradient descent

With the vigorous development of artificial intelligence technology, var...

0 Jinhuan Duan, et al. ∙

∙ 09/28/2018

Fluctuation-dissipation relations for stochastic gradient descent

The notion of the stationary equilibrium ensemble has played a central r...

0 Sho Yaida, et al. ∙

∙ 08/24/2020

Stochastic Gradient Descent Works Really Well for Stress Minimization

Stress minimization is among the best studied force-directed graph layou...

0 Katharina Börsig, et al. ∙