SphereFed: Hyperspherical Federated Learning

07/19/2022
by   Xin Dong, et al.
2

Federated Learning aims at training a global model from multiple decentralized devices (i.e. clients) without exchanging their private local data. A key challenge is the handling of non-i.i.d. (independent identically distributed) data across multiple clients that may induce disparities of their local features. We introduce the Hyperspherical Federated Learning (SphereFed) framework to address the non-i.i.d. issue by constraining learned representations of data points to be on a unit hypersphere shared by clients. Specifically, all clients learn their local representations by minimizing the loss with respect to a fixed classifier whose weights span the unit hypersphere. After federated training in improving the global model, this classifier is further calibrated with a closed-form solution by minimizing a mean squared loss. We show that the calibration solution can be computed efficiently and distributedly without direct access of local data. Extensive experiments indicate that our SphereFed approach is able to improve the accuracy of multiple existing federated learning algorithms by a considerable margin (up to 6 communication efficiency across datasets and model architectures.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/19/2020

Survey of Personalization Techniques for Federated Learning

Federated learning enables machine learning models to learn from private...
research
10/11/2021

Dual Attention-Based Federated Learning for Wireless Traffic Prediction

Wireless traffic prediction is essential for cellular networks to realiz...
research
05/01/2023

Scalable Data Point Valuation in Decentralized Learning

Existing research on data valuation in federated and swarm learning focu...
research
08/07/2023

Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Data

Federated Learning aims to learn a global model on the server side that ...
research
05/03/2021

OCTOPUS: Overcoming Performance andPrivatization Bottlenecks in Distributed Learning

The diversity and quantity of the data warehousing, gathering data from ...
research
06/27/2020

Federated Mutual Learning

Federated learning enables collaboratively training machine learning mod...
research
09/02/2022

Exact Decomposition of Quantum Channels for Non-IID Quantum Federated Learning

Federated learning refers to the task of performing machine learning wit...

Please sign up or login with your details

Forgot password? Click here to reset