: Downlink Compression for Cross-Device Federated Learning

02/01/2023
by   Ron Dorfman, et al.
0

Many compression techniques have been proposed to reduce the communication overhead of Federated Learning training procedures. However, these are typically designed for compressing model updates, which are expected to decay throughout training. As a result, such methods are inapplicable to downlink (i.e., from the parameter server to clients) compression in the cross-device setting, where heterogeneous clients may appear only once during training and thus must download the model parameters. In this paper, we propose a new framework () for downlink compression in the cross-device federated learning setting. Importantly, can be seamlessly combined with many uplink compression schemes, rendering it suitable for bi-directional compression. Through extensive evaluation, we demonstrate that offers significant bi-directional bandwidth reduction while achieving competitive accuracy to that of without compression.

READ FULL TEXT
research
03/07/2019

Robust and Communication-Efficient Federated Learning from Non-IID Data

Federated Learning allows multiple parties to jointly train a deep learn...
research
11/08/2020

Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning

With more regulations tackling users' privacy-sensitive data protection ...
research
12/11/2022

ResFed: Communication Efficient Federated Learning by Transmitting Deep Compressed Residuals

Federated learning enables cooperative training among massively distribu...
research
02/16/2023

THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression

Deep neural networks (DNNs) are the de-facto standard for essential use ...
research
12/05/2021

Intrinisic Gradient Compression for Federated Learning

Federated learning is a rapidly-growing area of research which enables a...
research
08/12/2022

A Fast Blockchain-based Federated Learning Framework with Compressed Communications

Recently, blockchain-based federated learning (BFL) has attracted intens...
research
12/30/2022

Deep Hierarchy Quantization Compression algorithm based on Dynamic Sampling

Unlike traditional distributed machine learning, federated learning stor...

Please sign up or login with your details

Forgot password? Click here to reset