Eduard Gorbunov | DeepAI

AI Chat

Featured Co-authors

Peter Richtarik
143 publications
Martin Takáč
53 publications
Gauthier Gidel
46 publications
Gennady Pekhimenko
37 publications
Fakhri Karray
37 publications
Zhize Li
27 publications
Nicolas Loizou
26 publications
Konstantin Mishchenko
26 publications
Alexander Gasnikov
25 publications
Samuel Horvath
22 publications
Pavel Dvurechensky
19 publications

∙ 05/30/2023

Clip21: Error Feedback for Gradient Clipping

Motivated by the increasing popularity and importance of large-scale tra...

0 Sarit Khirirat, et al. ∙

∙ 05/29/2023

Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity

We present a partially personalized formulation of Federated Learning (F...

0 Konstantin Mishchenko, et al. ∙

∙ 03/29/2023

Unified analysis of SGD-type methods

This note focuses on a simple approach to the unified analysis of SGD-ty...

0 Eduard Gorbunov, et al. ∙

∙ 03/08/2023

Byzantine-Robust Loopless Stochastic Variance-Reduced Gradient

Distributed optimization with open collaboration is a popular field sinc...

0 Nikita Fedin, et al. ∙

∙ 02/27/2023

Single-Call Stochastic Extragradient Methods for Structured Non-monotone Variational Inequalities: Improved Analysis under Weaker Conditions

Single-call stochastic extragradient methods, like stochastic past extra...

5 Sayantan Choudhury, et al. ∙

∙ 06/01/2022

Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top

Byzantine-robustness has been gaining a lot of attention due to the grow...

0 Eduard Gorbunov, et al. ∙

∙ 02/15/2022

Stochastic Gradient Descent-Ascent: Unified Theory and New Efficient Methods

Stochastic Gradient Descent-Ascent (SGDA) is one of the most prominent a...

8 Aleksandr Beznosikov, et al. ∙

∙ 02/02/2022

3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation

We propose and study a new class of gradient communication mechanisms fo...

2 Peter Richtarik, et al. ∙

∙ 12/20/2021

Distributed and Stochastic Optimization Methods with Gradient Compression and Local Steps

In this thesis, we propose new theoretical frameworks for the analysis o...

0 Eduard Gorbunov, et al. ∙

∙ 11/16/2021

Stochastic Extragradient: General Analysis and Improved Rates

The Stochastic Extragradient (SEG) method is one of the most popular alg...

10 Eduard Gorbunov, et al. ∙

∙ 10/08/2021

Extragradient Method: O(1/K) Last-Iterate Convergence for Monotone Variational Inequalities and Connections With Cocoercivity

Extragradient method (EG) Korpelevich [1976] is one of the most popular ...

0 Eduard Gorbunov, et al. ∙

∙ 10/07/2021

EF21 with Bells Whistles: Practical Algorithmic Extensions of Modern Error Feedback

First proposed by Seide (2014) as a heuristic, error feedback (EF) is a ...

7 Ilyas Fatkhullin, et al. ∙

∙ 06/21/2021

Secure Distributed Training at Scale

Some of the hardest problems in deep learning can be solved with the com...

5 Eduard Gorbunov, et al. ∙

∙ 06/10/2021

Near-Optimal High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise

Thanks to their practical efficiency and random nature of the data, stoc...

0 Eduard Gorbunov, et al. ∙

∙ 03/04/2021

Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices

Training deep neural networks on large datasets can often be accelerated...

0 Max Ryabinin, et al. ∙

∙ 02/15/2021

MARINA: Faster Non-Convex Distributed Learning with Compression

We develop and analyze MARINA: a new communication efficient method for ...

6 Eduard Gorbunov, et al. ∙

∙ 12/11/2020

Recent Theoretical Advances in Non-Convex Optimization

Motivated by recent increased interest in optimization algorithms for no...

0 Marina Danilova, et al. ∙

∙ 11/03/2020

Local SGD: Unified Theory and New Efficient Methods

We present a unified framework for analyzing local SGD methods in the co...

62 Eduard Gorbunov, et al. ∙

∙ 10/23/2020

Linearly Converging Error Compensated SGD

In this paper, we propose a unified analysis of variants of distributed ...

1 Eduard Gorbunov, et al. ∙

∙ 05/21/2020

Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping

In this paper, we propose a new accelerated stochastic first-order metho...

0 Eduard Gorbunov, et al. ∙

∙ 05/27/2019

A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent

In this paper we introduce a unified analysis of a large family of varia...

0 Eduard Gorbunov, et al. ∙

∙ 01/26/2019

Distributed Learning with Compressed Gradient Differences

Training very large machine learning models requires a distributed compu...

0 Konstantin Mishchenko, et al. ∙

∙ 04/08/2018

An Accelerated Directional Derivative Method for Smooth Stochastic Convex Optimization

We consider smooth stochastic convex optimization problems in the contex...

0 Pavel Dvurechensky, et al. ∙

∙ 02/25/2018

An Accelerated Method for Derivative-Free Smooth Stochastic Convex Optimization

We consider an unconstrained problem of minimization of a smooth convex ...

0 Pavel Dvurechensky, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world