FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems

04/15/2021
by   Li Li, et al.
0

Federated Learning (FL) is a novel distributed machine learning which allows thousands of edge devices to train model locally without uploading data concentrically to the server. But since real federated settings are resource-constrained, FL is encountered with systems heterogeneity which causes a lot of stragglers directly and then leads to significantly accuracy reduction indirectly. To solve the problems caused by systems heterogeneity, we introduce a novel self-adaptive federated framework FedSAE which adjusts the training task of devices automatically and selects participants actively to alleviate the performance degradation. In this work, we 1) propose FedSAE which leverages the complete information of devices' historical training tasks to predict the affordable training workloads for each device. In this way, FedSAE can estimate the reliability of each device and self-adaptively adjust the amount of training load per client in each round. 2) combine our framework with Active Learning to self-adaptively select participants. Then the framework accelerates the convergence of the global model. In our framework, the server evaluates devices' value of training based on their training loss. Then the server selects those clients with bigger value for the global model to reduce communication overhead. The experimental result indicates that in a highly heterogeneous system, FedSAE converges faster than FedAvg, the vanilla FL framework. Furthermore, FedSAE outperforms than FedAvg on several federated datasets - FedSAE improves test accuracy by 26.7 90.3

READ FULL TEXT

page 2

page 4

page 5

page 6

page 7

page 8

page 9

page 10

01/25/2020

TiFL: A Tier-based Federated Learning System

Federated Learning (FL) enables learning a shared model across many clie...
08/19/2022

Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning

Federated learning (FL) is a framework for machine learning across heter...
11/16/2021

HADFL: Heterogeneity-aware Decentralized Federated Learning Framework

Federated learning (FL) supports training models on geographically distr...
08/04/2022

ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity

When the available hardware cannot meet the memory and compute requireme...
04/07/2022

FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity

Federated Learning (FL) is an emerging framework for distributed process...
07/28/2020

Flower: A Friendly Federated Learning Research Framework

Federated Learning (FL) has emerged as a promising technique for edge de...
06/21/2022

FedHiSyn: A Hierarchical Synchronous Federated Learning Framework for Resource and Data Heterogeneity

Federated Learning (FL) enables training a global model without sharing ...