Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning

10/23/2019
by   Anis Elgabli, et al.
0

In this paper, we propose a communication-efficient decentralized machine learning (ML) algorithm, coined quantized group ADMM (Q-GADMM). Every worker in Q-GADMM communicates only with two neighbors, and updates its model via the alternating direct method of multiplier (ADMM), thereby ensuring fast convergence while reducing the number of communication rounds. Furthermore, each worker quantizes its model updates before transmissions, thereby decreasing the communication payload sizes. We prove that Q-GADMM converges for convex loss functions, and numerically show that Q-GADMM yields 7x less communication cost while achieving almost the same accuracy and convergence speed compared to a baseline without quantization, group ADMM (GADMM).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/14/2020

Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM

In this paper, we propose a communication-efficiently decentralized mach...
research
08/30/2019

GADMM: Fast and Communication Efficient Framework for Distributed Machine Learning

When the data is distributed across multiple servers, efficient data exc...
research
02/05/2022

Communication Efficient Federated Learning via Ordered ADMM in a Fully Decentralized Setting

The challenge of communication-efficient distributed optimization has at...
research
11/09/2019

L-FGADMM: Layer-Wise Federated Group ADMM for Communication Efficient Decentralized Deep Learning

This article proposes a communication-efficient decentralized deep learn...
research
10/14/2017

Robust Federated Learning Using ADMM in the Presence of Data Falsifying Byzantines

In this paper, we consider the problem of federated (or decentralized) l...
research
08/04/2022

QC-ODKLA: Quantized and Communication-Censored Online Decentralized Kernel Learning via Linearized ADMM

This paper focuses on online kernel learning over a decentralized networ...
research
05/28/2019

Distributed Linear Model Clustering over Networks: A Tree-Based Fused-Lasso ADMM Approach

In this work, we consider to improve the model estimation efficiency by ...

Please sign up or login with your details

Forgot password? Click here to reset