Fed-Focal Loss for imbalanced data classification in Federated Learning

11/12/2020
by   Dipankar Sarkar, et al.
0

The Federated Learning setting has a central server coordinating the training of a model on a network of devices. One of the challenges is variable training performance when the dataset has a class imbalance. In this paper, we address this by introducing a new loss function called Fed-Focal Loss. We propose to address the class imbalance by reshaping cross-entropy loss such that it down-weights the loss assigned to well-classified examples along the lines of focal loss. Additionally, by leveraging a tunable sampling framework, we take into account selective client model contributions on the central server to further focus the detector during training and hence improve its robustness. Using a detailed experimental analysis with the VIRTUAL (Variational Federated Multi-Task Learning) approach, we demonstrate consistently superior performance in both the balanced and unbalanced scenarios for MNIST, FEMNIST, VSN and HAR benchmarks. We obtain a more than 9 unbalanced MNIST benchmark. We further show that our technique can be adopted across multiple Federated Learning algorithms to get improvements.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/14/2019

Variational Federated Multi-Task Learning

In classical federated learning a central server coordinates the trainin...
research
09/14/2020

Fed+: A Family of Fusion Algorithms for Federated Learning

We present a class of methods for federated learning, which we call Fed+...
research
09/30/2022

Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction

Due to limited communication capacities of edge devices, most existing f...
research
11/14/2020

CatFedAvg: Optimising Communication-efficiency and Classification Accuracy in Federated Learning

Federated learning has allowed the training of statistical models over r...
research
05/27/2022

FedControl: When Control Theory Meets Federated Learning

To date, the most popular federated learning algorithms use coordinate-w...
research
06/02/2022

Federated Learning with a Sampling Algorithm under Isoperimetry

Federated learning uses a set of techniques to efficiently distribute th...
research
03/26/2019

Federated Learning Of Out-Of-Vocabulary Words

We demonstrate that a character-level recurrent neural network is able t...

Please sign up or login with your details

Forgot password? Click here to reset