CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning

11/14/2020
by   Dipankar Sarkar, et al.
0

Federated learning has allowed the training of statistical models over remote devices without the transfer of raw client data. In practice, training in heterogeneous and large networks introduce novel challenges in various aspects like network load, quality of client data, security and privacy. Recent works in FL have worked on improving communication efficiency and addressing uneven client data distribution independently, but none have provided a unified solution for both challenges. We introduce a new family of Federated Learning algorithms called CatFedAvg which not only improves the communication efficiency but improves the quality of learning using a category coverage maximization strategy. We use the FedAvg framework and introduce a simple and efficient step every epoch to collect meta-data about the client's training data structure which the central server uses to request a subset of weight updates. We explore two distinct variations which allow us to further explore the tradeoffs between communication efficiency and model accuracy. Our experiments based on a vision classification task have shown that an increase of 10 accuracy using the MNIST dataset with 70 transfer over FedAvg. We also run similar experiments with Fashion MNIST, KMNIST-10, KMNIST-49 and EMNIST-47. Further, under extreme data imbalance experiments for both globally and individual clients, we see the model performing better than FedAvg. The ablation study further explores its behaviour under varying data and client parameter conditions showcasing the robustness of the proposed approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/30/2020

Communication-Efficient Federated Learning via Optimal Client Sampling

Federated learning is a private and efficient framework for learning mod...
08/22/2021

Flexible Clustered Federated Learning for Client-Level Data Distribution Shift

Federated Learning (FL) enables the multiple participating devices to co...
07/12/2022

Federated Unlearning: How to Efficiently Erase a Client in FL?

With privacy legislation empowering users with the right to be forgotten...
11/12/2020

Fed-Focal Loss for imbalanced data classification in Federated Learning

The Federated Learning setting has a central server coordinating the tra...
04/01/2021

Federated Few-Shot Learning with Adversarial Learning

We are interested in developing a unified machine learning model over ma...
03/18/2020

Federated Visual Classification with Real-World Data Distribution

Federated Learning enables visual models to be trained on-device, bringi...
04/01/2022

Optimising Communication Overhead in Federated Learning Using NSGA-II

Federated learning is a training paradigm according to which a server-ba...