Federated Learning with Unbiased Gradient Aggregation and Controllable Meta Updating

10/18/2019
by   Xin Yao, et al.
0

Federated Averaging (FedAvg) serves as the fundamental framework in Federated Learning (FL) settings. However, we argue that 1) the multiple steps of local updating will result in gradient biases and 2) there is an inconsistency between the target distribution and the optimization objectives following the training paradigm in FedAvg. To tackle these problems, we first propose an unbiased gradient aggregation algorithm with the keep-trace gradient descent and gradient evaluation strategy. Then we introduce a meta updating procedure with a controllable meta training set to provide a clear and consistent optimization objective. Experimental results demonstrate that the proposed methods outperform compared ones with various network architectures in both the IID and non-IID FL settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/15/2021

FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning

We address the relatively unexplored problem of hyper-parameter optimiza...
research
08/17/2020

Inverse Distance Aggregation for Federated Learning with Non-IID Data

Federated learning (FL) has been a promising approach in the field of me...
research
05/26/2022

QUIC-FL: Quick Unbiased Compression for Federated Learning

Distributed Mean Estimation (DME) is a fundamental building block in com...
research
05/11/2021

FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis

Federated Learning (FL) is an emerging learning scheme that allows diffe...
research
08/06/2020

Federated Transfer Learning with Dynamic Gradient Aggregation

In this paper, a Federated Learning (FL) simulation platform is introduc...
research
09/27/2019

Improving Federated Learning Personalization via Model Agnostic Meta Learning

Federated Learning (FL) refers to learning a high quality global model b...
research
05/25/2023

Federated Composite Saddle Point Optimization

Federated learning (FL) approaches for saddle point problems (SPP) have ...

Please sign up or login with your details

Forgot password? Click here to reset