Towards Class Imbalance in Federated Learning

08/14/2020
by   Lixu Wang, et al.
16

Federated learning (FL) is a promising approach for training decentralized data located on local client devices while improving efficiency and privacy. However, the distribution and quantity of the training data on the clients' side may lead to significant challenges such as data imbalance and non-IID (non-independent and identically distributed) data, which could greatly impact the performance of the common model. While much effort has been devoted to helping FL models converge when encountering non-IID data, the imbalance issue has not been sufficiently addressed. In particular, as FL training is executed by exchanging gradients in an encrypted form, the training data is not completely observable to either clients or server, and previous methods for data imbalance do not perform well for FL. Therefore, it is crucial to design new methods for detecting data imbalance in FL and mitigating its impact. In this work, we propose a monitoring scheme that can infer the composition proportion of training data for each FL round, and design a new loss function – Ratio Loss to mitigate the impact of the imbalance. Our experiments demonstrate the importance of detecting data imbalance and taking measures as early as possible in FL training, and the effectiveness of our method in mitigating the impact. Our method is shown to significantly outperform previous methods, while maintaining client privacy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/02/2021

FedProf: Optimizing Federated Learning with Dynamic Data Profiling

Federated Learning (FL) has shown great potential as a privacy-preservin...
research
03/05/2022

Acceleration of Federated Learning with Alleviated Forgetting in Local Training

Federated learning (FL) enables distributed optimization of machine lear...
research
01/31/2023

Truthful Incentive Mechanism for Federated Learning with Crowdsourced Data Labeling

Federated learning (FL) has emerged as a promising paradigm that trains ...
research
09/30/2022

Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction

Due to limited communication capacities of edge devices, most existing f...
research
10/14/2019

Eavesdrop the Composition Proportion of Training Labels in Federated Learning

Federated learning (FL) has recently emerged as a new form of collaborat...
research
03/13/2023

Cross-device Federated Learning for Mobile Health Diagnostics: A First Study on COVID-19 Detection

Federated learning (FL) aided health diagnostic models can incorporate d...
research
04/06/2023

Learning Cautiously in Federated Learning with Noisy and Heterogeneous Clients

Federated learning (FL) is a distributed framework for collaboratively t...

Please sign up or login with your details

Forgot password? Click here to reset