DeepAI AI Chat
Log In Sign Up

FedCM: Federated Learning with Client-level Momentum

06/21/2021
by   Jing Xu, et al.
15

Federated Learning is a distributed machine learning approach which enables model training without data sharing. In this paper, we propose a new federated learning algorithm, Federated Averaging with Client-level Momentum (FedCM), to tackle problems of partial participation and client heterogeneity in real-world federated learning applications. FedCM aggregates global gradient information in previous communication rounds and modifies client gradient descent with a momentum-like term, which can effectively correct the bias and improve the stability of local SGD. We provide theoretical analysis to highlight the benefits of FedCM. We also perform extensive empirical studies and demonstrate that FedCM achieves superior performance in various tasks and is robust to different levels of client numbers, participation rate and client heterogeneity.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/08/2021

Double Momentum SGD for Federated Learning

Communication efficiency is crucial in federated learning. Conducting ma...
12/14/2022

FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation

The statistical heterogeneity of the non-independent and identically dis...
09/06/2020

Blockchain-based Federated Learning for Device Failure Detection in Industrial IoT

Device failure detection is one of most essential problems in industrial...
08/08/2020

Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning

Federated learning is a challenging optimization problem due to the hete...
02/12/2020

Federated Clustering via Matrix Factorization Models: From Model Averaging to Gradient Sharing

Recently, federated learning (FL) has drawn significant attention due to...
05/25/2022

Federated Adaptation of Reservoirs via Intrinsic Plasticity

We propose a novel algorithm for performing federated learning with Echo...
06/20/2022

Mitigating Data Heterogeneity in Federated Learning with Data Augmentation

Federated Learning (FL) is a prominent framework that enables training a...