Efficient and Light-Weight Federated Learning via Asynchronous Distributed Dropout

10/28/2022
by   Chen Dun, et al.
0

Asynchronous learning protocols have regained attention lately, especially in the Federated Learning (FL) setup, where slower clients can severely impede the learning process. Herein, we propose , a novel asynchronous FL framework that utilizes dropout regularization to handle device heterogeneity in distributed settings. Overall, achieves better performance compared to state of the art asynchronous methodologies, while resulting in less communication and training time overheads. The key idea revolves around creating “submodels” out of the global model, and distributing their training to workers, based on device heterogeneity. We rigorously justify that such an approach can be theoretically characterized. We implement our approach and compare it against other asynchronous baselines, both by design and by adapting existing synchronous FL algorithms to asynchronous scenarios. Empirically, reduces the communication cost and training time, while matching or improving the final test accuracy in diverse non-i.i.d. FL scenarios.

READ FULL TEXT
research
11/08/2021

Papaya: Practical, Private, and Scalable Federated Learning

Cross-device Federated Learning (FL) is a distributed learning paradigm ...
research
10/12/2020

FedAT: A Communication-Efficient Federated Learning Method with Asynchronous Tiers under Non-IID Data

Federated learning (FL) involves training a model over massive distribut...
research
03/02/2022

Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data

Federated learning (FL) is an emerging privacy-preserving paradigm that ...
research
11/10/2022

FedLesScan: Mitigating Stragglers in Serverless Federated Learning

Federated Learning (FL) is a machine learning paradigm that enables the ...
research
06/09/2022

Mobility Improves the Convergence of Asynchronous Federated Learning

This paper studies asynchronous Federated Learning (FL) subject to clien...
research
03/02/2021

Adaptive Transmission Scheduling in Wireless Networks for Asynchronous Federated Learning

In this paper, we study asynchronous federated learning (FL) in a wirele...
research
09/01/2021

Asynchronous Federated Learning for Sensor Data with Concept Drift

Federated learning (FL) involves multiple distributed devices jointly tr...

Please sign up or login with your details

Forgot password? Click here to reset