Integrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning

01/25/2023
by   Wenkai Yang, et al.
0

Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively in a data privacy-preserving manner. However, the data samples usually follow a long-tailed distribution in the real world, and FL on the decentralized and long-tailed data yields a poorly-behaved global model severely biased to the head classes with the majority of the training samples. To alleviate this issue, decoupled training has recently been introduced to FL, considering it has achieved promising results in centralized long-tailed learning by re-balancing the biased classifier after the instance-balanced training. However, the current study restricts the capacity of decoupled training in federated long-tailed learning with a sub-optimal classifier re-trained on a set of pseudo features, due to the unavailability of a global balanced dataset in FL. In this work, in order to re-balance the classifier more effectively, we integrate the local real data with the global gradient prototypes to form the local balanced datasets, and thus re-balance the classifier during the local training. Furthermore, we introduce an extra classifier in the training phase to help model the global data distribution, which addresses the problem of contradictory optimization goals caused by performing classifier re-balancing locally. Extensive experiments show that our method consistently outperforms the existing state-of-the-art methods in various settings.

READ FULL TEXT
research
04/28/2022

Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features

Federated learning (FL) provides a privacy-preserving solution for distr...
research
08/31/2021

GRP-FED: Addressing Client Imbalance in Federated Learning via Global-Regularized Personalization

Since data is presented long-tailed in reality, it is challenging for Fe...
research
06/30/2022

Towards Federated Long-Tailed Learning

Data privacy and class imbalance are the norm rather than the exception ...
research
05/26/2022

FedAug: Reducing the Local Learning Bias Improves Federated Learning on Heterogeneous Data

Federated Learning (FL) is a machine learning paradigm that learns from ...
research
03/11/2023

Stabilizing and Improving Federated Learning with Non-IID Data and Client Dropout

The label distribution skew induced data heterogeniety has been shown to...
research
06/17/2022

Open-Sampling: Exploring Out-of-Distribution data for Re-balancing Long-tailed datasets

Deep neural networks usually perform poorly when the training dataset su...
research
03/26/2021

Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentation

A thorough and holistic scene understanding is crucial for autonomous ve...

Please sign up or login with your details

Forgot password? Click here to reset