HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

10/03/2020
by   Enmao Diao, et al.
0

Federated Learning (FL) is a method of training machine learning models on private data distributed over a large number of possibly heterogeneous clients such as mobile phones and IoT devices. In this work, we propose a new federated learning framework named HeteroFL to address heterogeneous clients equipped with very different computation and communication capabilities. Our solution can enable the training of heterogeneous local models with varying computation complexities and still produce a single global inference model. For the first time, our method challenges the underlying assumption of existing work that local models have to share the same architecture as the global model. We demonstrate several strategies to enhance FL training and conduct extensive empirical evaluations, including five computation complexity levels of three model architecture on three datasets. We show that adaptively distributing subnetworks according to clients' capabilities is both computation and communication efficient.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/29/2021

FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorization

The underlying assumption of recent federated learning (FL) paradigms is...
research
06/03/2022

Supernet Training for Federated Image Classification under System Heterogeneity

Efficient deployment of deep neural networks across many devices and res...
research
03/08/2023

Memory-adaptive Depth-wise Heterogenous Federated Learning

Federated learning is a promising paradigm that allows multiple clients ...
research
08/21/2023

FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning

Recently, foundation models have exhibited remarkable advancements in mu...
research
10/27/2022

Exploiting Features and Logits in Heterogeneous Federated Learning

Due to the rapid growth of IoT and artificial intelligence, deploying ne...
research
09/15/2021

Federated Learning of Molecular Properties in a Heterogeneous Setting

Chemistry research has both high material and computational costs to con...

Please sign up or login with your details

Forgot password? Click here to reset