research
∙
05/30/2023
Shuffle SGD is Always Better than SGD: Improved Analysis of SGD with Arbitrary Data Orders
Stochastic Gradient Descent (SGD) algorithms are widely used in optimizi...
share
research
∙
05/02/2023
Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees
Gradient clipping is a popular modification to standard (stochastic) gra...
research
∙
02/02/2023
Convergence of Gradient Descent with Linearly Correlated Noise and Applications to Differentially Private Learning
We study stochastic optimization with linearly correlated noise. Our stu...
research
∙
01/03/2023
Decentralized Gradient Tracking with Local Steps
Gradient tracking (GT) is an algorithm designed for solving decentralize...
research
∙
06/16/2022
Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning
We study the asynchronous stochastic gradient descent algorithm for dist...
research
∙
04/13/2022
Data-heterogeneity-aware Mixing for Decentralized Learning
Decentralized learning provides an effective framework to train machine ...
research
∙
02/08/2022
An Improved Analysis of Gradient Tracking for Decentralized Machine Learning
We consider decentralized machine learning over a network where the trai...
research
∙
10/08/2021
RelaySum for Decentralized Deep Learning on Heterogeneous Data
In decentralized machine learning, workers compute model updates on thei...
research
∙
06/15/2021
Decentralized Local Stochastic Extra-Gradient for Variational Inequalities
We consider decentralized stochastic variational inequalities where the ...
research
∙
02/09/2021
Consensus Control for Decentralized Deep Learning
Decentralized training of deep learning models enables on-device learnin...
research
∙
11/03/2020
A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!
Decentralized optimization methods enable on-device training of machine ...
research
∙
03/23/2020
A Unified Theory of Decentralized SGD with Changing Topology and Local Updates
Decentralized stochastic optimization methods have gained a lot of atten...
research
∙
07/22/2019
Decentralized Deep Learning with Arbitrary Communication Compression
Decentralized training of deep learning models is a key element for enab...
research
∙
02/01/2019
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication
We consider decentralized stochastic optimization with the objective fun...
research
∙
10/16/2018