Compressed Gradient Tracking Methods for Decentralized Optimization with Linear Convergence

03/25/2021
by   Yiwei Liao, et al.
0

Communication compression techniques are of growing interests for solving the decentralized optimization problem under limited communication, where the global objective is to minimize the average of local cost functions over a multi-agent network using only local computation and peer-to-peer communication. In this paper, we first propose a novel compressed gradient tracking algorithm (C-GT) that combines gradient tracking technique with communication compression. In particular, C-GT is compatible with a general class of compression operators that unifies both unbiased and biased compressors. We show that C-GT inherits the advantages of gradient tracking-based algorithms and achieves linear convergence rate for strongly convex and smooth objective functions. In the second part of this paper, we propose an error feedback based compressed gradient tracking algorithm (EF-C-GT) to further improve the algorithm efficiency for biased compression operators. Numerical examples complement the theoretical findings and demonstrate the efficiency and flexibility of the proposed algorithms.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/14/2021

Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks

In this paper, we propose two communication-efficient algorithms for dec...
research
12/23/2021

Decentralized Multi-Task Stochastic Optimization With Compressed Communications

We consider a multi-agent network where each node has a stochastic (loca...
research
05/14/2021

Innovation Compression for Communication-efficient Distributed Optimization with Linear Convergence

Information compression is essential to reduce communication cost in dis...
research
02/01/2022

DoCoM-SGT: Doubly Compressed Momentum-assisted Stochastic Gradient Tracking Algorithm for Communication Efficient Decentralized Learning

This paper proposes the Doubly Compressed Momentum-assisted Stochastic G...
research
12/03/2021

A Divide-and-Conquer Algorithm for Distributed Optimization on Networks

In this paper, we consider networks with topologies described by some co...
research
09/03/2020

Distributed Online Optimization via Gradient Tracking with Adaptive Momentum

This paper deals with a network of computing agents aiming to solve an o...
research
06/12/2023

On the Computation-Communication Trade-Off with A Flexible Gradient Tracking Approach

We propose a flexible gradient tracking approach with adjustable computa...

Please sign up or login with your details

Forgot password? Click here to reset