Adaptive Configuration for Heterogeneous Participants in Decentralized Federated Learning

12/05/2022
by   Yunming Liao, et al.
0

Data generated at the network edge can be processed locally by leveraging the paradigm of edge computing (EC). Aided by EC, decentralized federated learning (DFL), which overcomes the single-point-of-failure problem in the parameter server (PS) based federated learning, is becoming a practical and popular approach for machine learning over distributed data. However, DFL faces two critical challenges, , system heterogeneity and statistical heterogeneity introduced by edge devices. To ensure fast convergence with the existence of slow edge devices, we present an efficient DFL method, termed FedHP, which integrates adaptive control of both local updating frequency and network topology to better support the heterogeneous participants. We establish a theoretical relationship between local updating frequency and network topology regarding model training performance and obtain a convergence upper bound. Upon this, we propose an optimization algorithm, that adaptively determines local updating frequencies and constructs the network topology, so as to speed up convergence and improve the model accuracy. Evaluation results show that the proposed FedHP can reduce the completion time by about 51 accuracy by at least 5 baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/20/2021

FedSkel: Efficient Federated Learning on Heterogeneous Systems with Skeleton Gradients Update

Federated learning aims to protect users' privacy while performing data ...
research
02/12/2020

Towards Federated Learning: Robustness Analytics to Data Heterogeneity

Federated Learning allows remote centralized server training models with...
research
10/15/2021

Nothing Wasted: Full Contribution Enforcement in Federated Edge Learning

The explosive amount of data generated at the network edge makes mobile ...
research
09/19/2022

Heterogeneous Federated Learning on a Graph

Federated learning, where algorithms are trained across multiple decentr...
research
03/16/2019

Practical Distributed Learning: Secure Machine Learning with Communication-Efficient Local Updates

Federated learning on edge devices poses new challenges arising from wor...
research
12/02/2022

Generative Data Augmentation for Non-IID Problem in Decentralized Clinical Machine Learning

Swarm learning (SL) is an emerging promising decentralized machine learn...
research
04/15/2021

D-Cliques: Compensating NonIIDness in Decentralized Federated Learning with Topology

The convergence speed of machine learning models trained with Federated ...

Please sign up or login with your details

Forgot password? Click here to reset