FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction

by   Liang Gao, et al.

Federated learning (FL) allows multiple clients to collectively train a high-performance global model without sharing their private data. However, the key challenge in federated learning is that the clients have significant statistical heterogeneity among their local data distributions, which would cause inconsistent optimized local models on the client-side. To address this fundamental dilemma, we propose a novel federated learning algorithm with local drift decoupling and correction (FedDC). Our FedDC only introduces lightweight modifications in the local training phase, in which each client utilizes an auxiliary local drift variable to track the gap between the local model parameter and the global model parameters. The key idea of FedDC is to utilize this learned local drift variable to bridge the gap, i.e., conducting consistency in parameter-level. The experiment results and analysis demonstrate that FedDC yields expediting convergence and better performance on various image classification tasks, robust in partial participation settings, non-iid data, and heterogeneous clients.


page 1

page 2

page 3

page 4


AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimation

In Federated Learning (FL), a number of clients or devices collaborate t...

Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters

Federated learning is a distributed, privacy-aware learning scenario whi...

HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images

Multiple medical institutions collaboratively training a model using fed...

FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning

Federated Distillation (FD) is a popular novel algorithmic paradigm for ...

Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning

Federated learning aims to train models collaboratively across different...

Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

Federated learning is a challenging optimization problem due to the hete...

Multi-Level Branched Regularization for Federated Learning

A critical challenge of federated learning is data heterogeneity and imb...