DeepAI
Log In Sign Up

Federated Learning with Nesterov Accelerated Gradient Momentum Method

09/18/2020
by   Zhengjie Yang, et al.
0

Federated learning (FL) is a fast-developing technique that allows multiple workers to train a global model based on a distributed dataset. Conventional FL employs gradient descent algorithm, which may not be efficient enough. It is well known that Nesterov Accelerated Gradient (NAG) is more advantageous in centralized training environment, but it is not clear how to quantify the benefits of NAG in FL so far. In this work, we focus on a version of FL based on NAG (FedNAG) and provide a detailed convergence analysis. The result is compared with conventional FL based on gradient descent. One interesting conclusion is that as long as the learning step size is sufficiently small, FedNAG outperforms FedAvg. Extensive experiments based on real-world datasets are conducted, verifying our conclusions and confirming the better convergence performance of FedNAG.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/08/2019

Accelerating Federated Learning via Momentum Gradient Descent

Federated learning (FL) provides a communication-efficient approach to s...
04/27/2021

Confined Gradient Descent: Privacy-preserving Optimization for Federated Learning

Federated learning enables multiple participants to collaboratively trai...
05/11/2021

FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis

Federated Learning (FL) is an emerging learning scheme that allows diffe...
10/25/2022

Federated Learning Using Variance Reduced Stochastic Gradient for Probabilistically Activated Agents

This paper proposes an algorithm for Federated Learning (FL) with a two-...
12/16/2020

FedADC: Accelerated Federated Learning with Drift Control

Federated learning (FL) has become de facto framework for collaborative ...
06/14/2021

Dynamic Gradient Aggregation for Federated Domain Adaptation

In this paper, a new learning algorithm for Federated Learning (FL) is i...
11/26/2021

An Optimization Framework for Federated Edge Learning

The optimal design of federated learning (FL) algorithms for solving gen...