
Differentially Private Accelerated Optimization Algorithms
We present two classes of differentially private optimization algorithms...
read it

Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo
Stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Ham...
read it

IDEAL: Inexact DEcentralized Accelerated Augmented Lagrangian Method
We introduce a framework for designing primal methods under the decentra...
read it

A Stochastic Subgradient Method for Distributionally Robust NonConvex Learning
We consider a distributionally robust formulation of stochastic optimiza...
read it

The HeavyTail Phenomenon in SGD
In recent years, various notions of capacity and complexity have been pr...
read it

Fractional momentpreserving initialization schemes for training fullyconnected neural networks
A common approach to initialization in deep neural networks is to sample...
read it

NonConvex Stochastic Optimization via NonReversible Stochastic Gradient Langevin Dynamics
Stochastic gradient Langevin dynamics (SGLD) is a poweful algorithm for ...
read it

Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under HeavyTailed Gradient Noise
Stochastic gradient descent with momentum (SGDm) is one of the most popu...
read it

On the HeavyTailed Theory of Stochastic Gradient Descent for Deep Neural Networks
The gradient noise (GN) in the stochastic gradient descent (SGD) algorit...
read it

First Exit Time Analysis of Stochastic Gradient Descent Under HeavyTailed Gradient Noise
Stochastic gradient descent (SGD) has been widely used in machine learni...
read it

A Universally Optimal Multistage Accelerated Stochastic Gradient Method
We study the problem of minimizing a strongly convex and smooth function...
read it

Accelerated Linear Convergence of Stochastic Momentum Methods in Wasserstein Distances
Momentum methods such as Polyak's heavy ball (HB) method, Nesterov's acc...
read it

A TailIndex Analysis of Stochastic Gradient Noise in Deep Neural Networks
The gradient noise (GN) in the stochastic gradient descent (SGD) algorit...
read it

Breaking Reversibility Accelerates Langevin Dynamics for Global NonConvex Optimization
Langevin dynamics (LD) has been proven to be a powerful technique for op...
read it

Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for NonConvex Stochastic Optimization: NonAsymptotic Performance Bounds and MomentumBased Acceleration
Stochastic gradient Hamiltonian Monte Carlo (SGHMC) is a variant of stoc...
read it

Avoiding Communication in Proximal Methods for Convex Optimization Problems
The fast iterative soft thresholding algorithm (FISTA) is used to solve ...
read it
Mert Gurbuzbalaban
is this you? claim profile