research
∙
05/30/2023
Clip21: Error Feedback for Gradient Clipping
Motivated by the increasing popularity and importance of large-scale tra...
share
research
∙
05/29/2023
Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity
We present a partially personalized formulation of Federated Learning (F...
research
∙
03/29/2023
Unified analysis of SGD-type methods
This note focuses on a simple approach to the unified analysis of SGD-ty...
research
∙
03/08/2023
Byzantine-Robust Loopless Stochastic Variance-Reduced Gradient
Distributed optimization with open collaboration is a popular field sinc...
research
∙
02/27/2023
Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditions
Single-call stochastic extragradient methods, like stochastic past extra...
research
∙
06/01/2022
Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top
Byzantine-robustness has been gaining a lot of attention due to the grow...
research
∙
02/15/2022
Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods
Stochastic Gradient Descent-Ascent (SGDA) is one of the most prominent a...
research
∙
02/02/2022
3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation
We propose and study a new class of gradient communication mechanisms fo...
research
∙
12/20/2021
Distributed and Stochastic Optimization Methods with Gradient Compression and Local Steps
In this thesis, we propose new theoretical frameworks for the analysis o...
research
∙
11/16/2021
Stochastic Extragradient: General Analysis and Improved Rates
The Stochastic Extragradient (SEG) method is one of the most popular alg...
research
∙
10/08/2021
Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivity
Extragradient method (EG) Korpelevich [1976] is one of the most popular ...
research
∙
10/07/2021
EF21 with Bells Whistles: Practical Algorithmic Extensions of Modern Error Feedback
First proposed by Seide (2014) as a heuristic, error feedback (EF) is a ...
research
∙
06/21/2021
Secure Distributed Training at Scale
Some of the hardest problems in deep learning can be solved with the com...
research
∙
06/10/2021
Near-Optimal High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise
Thanks to their practical efficiency and random nature of the data, stoc...
research
∙
03/04/2021
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices
Training deep neural networks on large datasets can often be accelerated...
research
∙
02/15/2021
MARINA: Faster Non-Convex Distributed Learning with Compression
We develop and analyze MARINA: a new communication efficient method for ...
research
∙
12/11/2020
Recent Theoretical Advances in Non-Convex Optimization
Motivated by recent increased interest in optimization algorithms for no...
research
∙
11/03/2020
Local SGD: Unified Theory and New Efficient Methods
We present a unified framework for analyzing local SGD methods in the co...
research
∙
10/23/2020
Linearly Converging Error Compensated SGD
In this paper, we propose a unified analysis of variants of distributed ...
research
∙
05/21/2020
Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping
In this paper, we propose a new accelerated stochastic first-order metho...
research
∙
05/27/2019
A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent
In this paper we introduce a unified analysis of a large family of varia...
research
∙
01/26/2019
Distributed Learning with Compressed Gradient Differences
Training very large machine learning models requires a distributed compu...
research
∙
04/08/2018
An Accelerated Directional Derivative Method for Smooth Stochastic Convex Optimization
We consider smooth stochastic convex optimization problems in the contex...
research
∙
02/25/2018