AsyncFedED: Asynchronous Federated Learning with Euclidean Distance based Adaptive Weight Aggregation

05/27/2022
by   Qiyuan Wang, et al.
0

In an asynchronous federated learning framework, the server updates the global model once it receives an update from a client instead of waiting for all the updates to arrive as in the synchronous setting. This allows heterogeneous devices with varied computing power to train the local models without pausing, thereby speeding up the training process. However, it introduces the stale model problem, where the newly arrived update was calculated based on a set of stale weights that are older than the current global model, which may hurt the convergence of the model. In this paper, we present an asynchronous federated learning framework with a proposed adaptive weight aggregation algorithm, referred to as AsyncFedED. To the best of our knowledge this aggregation method is the first to take the staleness of the arrived gradients, measured by the Euclidean distance between the stale model and the current global model, and the number of local epochs that have been performed, into account. Assuming general non-convex loss functions, we prove the convergence of the proposed method theoretically. Numerical results validate the effectiveness of the proposed AsyncFedED in terms of the convergence rate and model accuracy compared to the existing methods for three considered tasks.

READ FULL TEXT
06/01/2023

CSMAAFL: Client Scheduling and Model Aggregation in Asynchronous Federated Learning

Asynchronous federated learning aims to solve the straggler problem in h...
04/08/2022

Global Update Guided Federated Learning

Federated learning protects data privacy and security by exchanging mode...
06/29/2022

AFAFed – Protocol analysis

In this paper, we design, analyze the convergence properties and address...
02/12/2021

Stragglers Are Not Disaster: A Hybrid Federated Learning Algorithm with Delayed Gradients

Federated learning (FL) is a new machine learning framework which trains...
04/28/2022

Improving the Robustness of Federated Learning for Severely Imbalanced Datasets

With the ever increasing data deluge and the success of deep neural netw...
03/12/2023

Asynchronous Decentralized Federated Lifelong Learning for Landmark Localization in Medical Imaging

Federated learning is a recent development in the machine learning area ...

Please sign up or login with your details

Forgot password? Click here to reset