FedGroup: Ternary Cosine Similarity-based Clustered Federated Learning Framework toward High Accuracy in Heterogeneous Data

10/14/2020
by   Moming Duan, et al.
0

Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training data locally. Unlike the centralized training setting, the non-IID and imbalanced (statistical heterogeneity) training data of FL is distributed in the federated network, which will increase the divergences between the local models and global model and further degrade the performance. In this paper, we propose a novel federated learning framework FedGroup based on a similarity-based clustering strategy, in which we 1) group the training of clients based on the similarities between the clients' optimize directions; 2) reduce the complexity of high-dimension low-sample size (HDLSS) parameter updates data clustering by decomposing the direction vectors to derive the ternary cosine similarity. FedGroup can achieve improvements by dividing joint optimization into groups of sub-optimization, and can be combined with FedProx, the state-of-the-art federated optimization algorithm. We evaluate FedGroup and FedGrouProx (combined with FedProx) on several open datasets. The experimental results show that our proposed frameworks significantly improving absolute test accuracy by +14.7 compared to FedProx.

READ FULL TEXT

page 2

page 3

page 4

page 5

page 6

page 7

page 8

page 9

research
08/22/2021

Flexible Clustered Federated Learning for Client-Level Data Distribution Shift

Federated Learning (FL) enables the multiple participating devices to co...
research
07/09/2021

Lithography Hotspot Detection via Heterogeneous Federated Learning with Local Adaptation

As technology scaling is approaching the physical limit, lithography hot...
research
06/15/2022

Adaptive Expert Models for Personalization in Federated Learning

Federated Learning (FL) is a promising framework for distributed learnin...
research
07/02/2019

Astraea: Self-balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications

Federated learning (FL) is a distributed deep learning method which enab...
research
06/24/2021

Personalized Federated Learning with Clustered Generalization

We study the recent emerging personalized federated learning (PFL) that ...
research
10/16/2021

Incentivize to Build: A Crowdsourcing Framework for Federated Learning

Federated learning (FL) rests on the notion of training a global model i...
research
05/21/2023

Confidence-aware Personalized Federated Learning via Variational Expectation Maximization

Federated Learning (FL) is a distributed learning scheme to train a shar...

Please sign up or login with your details

Forgot password? Click here to reset