Communication-Efficient Federated Learning via Predictive Coding

by   Kai Yue, et al.

Federated learning can enable remote workers to collaboratively train a shared machine learning model while allowing training data to be kept locally. In the use case of wireless mobile devices, the communication overhead is a critical bottleneck due to limited power and bandwidth. Prior work has utilized various data compression tools such as quantization and sparsification to reduce the overhead. In this paper, we propose a predictive coding based communication scheme for federated learning. The scheme has shared prediction functions among all devices and allows each worker to transmit a compressed residual vector derived from the reference. In each communication round, we select the predictor and quantizer based on the rate-distortion cost, and further reduce the redundancy with entropy coding. Extensive simulations reveal that the communication cost can be reduced up to 99 performance when compared with other baseline methods.



There are no comments yet.


page 8


Federated Learning via Plurality Vote

Federated learning allows collaborative workers to solve a machine learn...

Toward Communication Efficient Adaptive Gradient Method

In recent years, distributed optimization is proven to be an effective a...

Optimizing the Communication-Accuracy Trade-off in Federated Learning with Rate-Distortion Theory

A significant bottleneck in federated learning is the network communicat...

ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training

Federated learning is a powerful distributed learning scheme that allows...

OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning

The diversity and quantity of the data warehousing, gathering data from ...

FedSynth: Gradient Compression via Synthetic Data in Federated Learning

Model compression is important in federated learning (FL) with large mod...

Resource Allocation in Mobility-Aware Federated Learning Networks: A Deep Reinforcement Learning Approach

Federated learning allows mobile devices, i.e., workers, to use their lo...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.