FedADC: Accelerated Federated Learning with Drift Control

by   Emre Ozfatura, et al.

Federated learning (FL) has become de facto framework for collaborative learning among edge devices with privacy concern. The core of the FL strategy is the use of stochastic gradient descent (SGD) in a distributed manner. Large scale implementation of FL brings new challenges, such as the incorporation of acceleration techniques designed for SGD into the distributed setting, and mitigation of the drift problem due to non-homogeneous distribution of local datasets. These two problems have been separately studied in the literature; whereas, in this paper, we show that it is possible to address both problems using a single strategy without any major alteration to the FL framework, or introducing additional computation and communication load. To achieve this goal, we propose FedADC, which is an accelerated FL algorithm with drift control. We empirically illustrate the advantages of FedADC.


page 1

page 2

page 3

page 4


FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning

We address the relatively unexplored problem of hyper-parameter optimiza...

Federated Learning with Nesterov Accelerated Gradient Momentum Method

Federated learning (FL) is a fast-developing technique that allows multi...

Evolution and trade-off dynamics of functional load

Function Load (FL) quantifies the contributions by phonological contrast...

Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

We study the optimization aspects of personalized Federated Learning (FL...

Asynchronous Federated Learning for Sensor Data with Concept Drift

Federated learning (FL) involves multiple distributed devices jointly tr...

Stochastic-Sign SGD for Federated Learning with Theoretical Guarantees

Federated learning (FL) has emerged as a prominent distributed learning ...

Federated Learning over Noisy Channels: Convergence Analysis and Design Examples

Does Federated Learning (FL) work when both uplink and downlink communic...