FedMD: Heterogenous Federated Learning via Model Distillation

10/08/2019
by   Daliang Li, et al.
0

Federated learning enables the creation of a powerful centralized model without compromising data privacy of multiple participants. While successful, it does not incorporate the case where each participant independently designs its own model. Due to intellectual property concerns and heterogeneous nature of tasks and data, this is a widespread requirement in applications of federated learning to areas such as health care and AI as a service. In this work, we use transfer learning and knowledge distillation to develop a universal framework that enables federated learning when each agent owns not only their private data, but also uniquely designed models. We test our framework on the MNIST/FEMNIST dataset and the CIFAR10/CIFAR100 dataset and observe fast improvement across all participating models. With 10 distinct participants, the final test accuracy of each model on average receives a 20 gain on top of what's possible without collaboration and is only a few percent lower than the performance each model would have obtained if all private datasets were pooled and made directly available for all participants.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/02/2022

FedDKD: Federated Learning with Decentralized Knowledge Distillation

The performance of federated learning in neural networks is generally in...
research
07/19/2022

FedX: Unsupervised Federated Learning with Cross Knowledge Distillation

This paper presents FedX, an unsupervised federated learning framework. ...
research
02/12/2020

Salvaging Federated Learning by Local Adaptation

Federated learning (FL) is a heavily promoted approach for training ML m...
research
10/28/2021

Towards Model Agnostic Federated Learning Using Knowledge Distillation

An often unquestioned assumption underlying most current federated learn...
research
06/07/2023

Resource Aware Clustering for Tackling the Heterogeneity of Participants in Federated Learning

Federated Learning is a training framework that enables multiple partici...
research
08/26/2022

Abnormal Local Clustering in Federated Learning

Federated learning is a model for privacy without revealing private data...
research
09/03/2022

Suppressing Noise from Built Environment Datasets to Reduce Communication Rounds for Convergence of Federated Learning

Smart sensing provides an easier and convenient data-driven mechanism fo...

Please sign up or login with your details

Forgot password? Click here to reset