HADFL: Heterogeneity-aware Decentralized Federated Learning Framework

11/16/2021
by   Jing Cao, et al.
0

Federated learning (FL) supports training models on geographically distributed devices. However, traditional FL systems adopt a centralized synchronous strategy, putting high communication pressure and model generalization challenge. Existing optimizations on FL either fail to speedup training on heterogeneous devices or suffer from poor communication efficiency. In this paper, we propose HADFL, a framework that supports decentralized asynchronous training on heterogeneous devices. The devices train model locally with heterogeneity-aware local steps using local data. In each aggregation cycle, they are selected based on probability to perform model synchronization and aggregation. Compared with the traditional FL system, HADFL can relieve the central server's communication pressure, efficiently utilize heterogeneous computing power, and can achieve a maximum speedup of 3.15x than decentralized-FedAvg and 4.68x than Pytorch distributed training scheme, respectively, with almost no loss of convergence accuracy.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/23/2021

Device Scheduling and Update Aggregation Policies for Asynchronous Federated Learning

Federated Learning (FL) is a newly emerged decentralized machine learnin...
research
06/21/2022

FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity

Federated Learning (FL) enables training a global model without sharing ...
research
08/20/2023

Arena: A Learning-based Synchronization Scheme for Hierarchical Federated Learning–Technical Report

Federated learning (FL) enables collaborative model training among distr...
research
11/23/2022

Event-Triggered Decentralized Federated Learning over Resource-Constrained Edge Devices

Federated learning (FL) is a technique for distributed machine learning ...
research
08/17/2022

NET-FLEET: Achieving Linear Convergence Speedup for Fully Decentralized Federated Learning with Heterogeneous Data

Federated learning (FL) has received a surge of interest in recent years...
research
06/29/2022

AFAFed – Protocol analysis

In this paper, we design, analyze the convergence properties and address...
research
02/04/2019

Hop: Heterogeneity-Aware Decentralized Training

Recent work has shown that decentralized algorithms can deliver superior...

Please sign up or login with your details

Forgot password? Click here to reset