Finite-Bit Quantization For Distributed Algorithms With Linear Convergence

07/23/2021
by   Chang-Shen Lee, et al.
0

This paper studies distributed algorithms for (strongly convex) composite optimization problems over mesh networks, subject to quantized communications. Instead of focusing on a specific algorithmic design, we propose a black-box model casting distributed algorithms in the form of fixed-point iterates, converging at linear rate. The algorithmic model is coupled with a novel (random) Biased Compression (BC-)rule on the quantizer design, which preserves linear convergence. A new quantizer coupled with a communication-efficient encoding scheme is also proposed, which efficiently implements the BC-rule using a finite number of bits. This contrasts with most of existing quantization rules, whose implementation calls for an infinite number of bits. A unified communication complexity analysis is developed for the black-box model, determining the average number of bit required to reach a solution of the optimization problem within the required accuracy. Numerical results validate our theoretical findings and show that distributed algorithms equipped with the proposed quantizer have more favorable communication complexity than algorithms using existing quantization rules.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

02/14/2021

Communication-Efficient Distributed Optimization with Quantized Preconditioners

We investigate fast and communication-efficient algorithms for the class...
02/17/2019

Quantized Frank-Wolfe: Communication-Efficient Distributed Optimization

How can we efficiently mitigate the overhead of gradient communications ...
06/19/2020

DEED: A General Quantization Scheme for Communication Efficiency in Bits

In distributed optimization, a popular technique to reduce communication...
02/05/2022

Distributed Learning With Sparsified Gradient Differences

A very large number of communications are typically required to solve di...
05/14/2021

Innovation Compression for Communication-efficient Distributed Optimization with Linear Convergence

Information compression is essential to reduce communication cost in dis...
05/23/2021

From Chaos to Pseudo-Randomness: A Case Study on the 2D Coupled Map Lattice

Applying chaos theory for secure digital communications is promising and...
03/09/2022

Correlated quantization for distributed mean estimation and optimization

We study the problem of distributed mean estimation and optimization und...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.