research
∙
05/20/2023
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent
Algorithmic stability is an important notion that has proven powerful fo...
share
research
∙
02/10/2023
Cyclic and Randomized Stepsizes Invoke Heavier Tails in SGD
Cyclic and randomized stepsizes are widely used in the deep learning pra...
research
∙
01/27/2023
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions
Heavy-tail phenomena in stochastic gradient descent (SGD) have been repo...
research
∙
11/29/2022
Penalized Langevin and Hamiltonian Monte Carlo Algorithms for Constrained Sampling
We consider the constrained sampling problem where the goal is to sample...
research
∙
06/02/2022
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares
Recent studies have shown that heavy tails can emerge in stochastic opti...
research
∙
05/13/2022
Heavy-Tail Phenomenon in Decentralized SGD
Recent theoretical studies have shown that heavy-tails can emerge in sto...
research
∙
06/09/2021
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms
Understanding generalization in deep learning has been one of the major ...
research
∙
02/20/2021
Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance
Recent studies have provided both empirical and theoretical evidence ill...
research
∙
02/13/2021
Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections
Gaussian noise injections (GNIs) are a family of simple and widely-used ...
research
∙
07/01/2020
Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
Stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Ham...
research
∙
06/08/2020
The Heavy-Tail Phenomenon in SGD
In recent years, various notions of capacity and complexity have been pr...
research
∙
04/06/2020
Non-Convex Stochastic Optimization via Non-Reversible Stochastic Gradient Langevin Dynamics
Stochastic gradient Langevin dynamics (SGLD) is a poweful algorithm for ...
research
∙
02/13/2020
Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise
Stochastic gradient descent with momentum (SGDm) is one of the most popu...
research
∙
01/22/2019
Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances
Momentum methods such as Polyak's heavy ball (HB) method, Nesterov's acc...
research
∙
12/19/2018
Breaking Reversibility Accelerates Langevin Dynamics for Global Non-Convex Optimization
Langevin dynamics (LD) has been proven to be a powerful technique for op...
research
∙
09/12/2018