FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction

03/22/2022
by   Liang Gao, et al.
4

Federated learning (FL) allows multiple clients to collectively train a high-performance global model without sharing their private data. However, the key challenge in federated learning is that the clients have significant statistical heterogeneity among their local data distributions, which would cause inconsistent optimized local models on the client-side. To address this fundamental dilemma, we propose a novel federated learning algorithm with local drift decoupling and correction (FedDC). Our FedDC only introduces lightweight modifications in the local training phase, in which each client utilizes an auxiliary local drift variable to track the gap between the local model parameter and the global model parameters. The key idea of FedDC is to utilize this learned local drift variable to bridge the gap, i.e., conducting consistency in parameter-level. The experiment results and analysis demonstrate that FedDC yields expediting convergence and better performance on various image classification tasks, robust in partial participation settings, non-iid data, and heterogeneous clients.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/27/2022

AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimation

In Federated Learning (FL), a number of clients or devices collaborate t...
12/30/2019

Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters

Federated learning is a distributed, privacy-aware learning scenario whi...
12/20/2021

HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images

Multiple medical institutions collaboratively training a model using fed...
02/04/2021

FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning

Federated Distillation (FD) is a popular novel algorithmic paradigm for ...
10/01/2022

Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning

Federated learning aims to train models collaboratively across different...
08/08/2020

Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

Federated learning is a challenging optimization problem due to the hete...
07/14/2022

Multi-Level Branched Regularization for Federated Learning

A critical challenge of federated learning is data heterogeneity and imb...