Federated Mutual Learning

06/27/2020
by   Tao Shen, et al.
0

Federated learning enables collaboratively training machine learning models on decentralized data. The three types of heterogeneous natures that is data, model, and objective bring about unique challenges to the canonical federated learning algorithm (FederatedAveraging), where one shared model is produced by and for all clients. First, due to the Non-IIDness of data, the global shared model may perform worse than local models that solely trained on their private data; Second, clients may need to design their own model because of different communication and computing abilities of devices, which is also private property that should be protected; Third, the objective of achieving consensus throughout the training process will compromise the personalities of clients. In this work, we present a novel federated learning paradigm, named Federated Mutual Leaning (FML), dealing with the three heterogeneities. FML allows clients designing their customized models and training independently, thus the Non-IIDness of data is no longer a bug but a feature that clients can be personally served better. Local customized models can benefit from collaboratively training without compromising personalities. Global model does not have to be an out-of-the-box (OOTB) product but a meta-learner which requires local adaptation for new participants. The experiments show that FML can achieve better performance, robustness and communication efficiency than alternatives.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/19/2020

Survey of Personalization Techniques for Federated Learning

Federated learning enables machine learning models to learn from private...
research
06/02/2023

Federated Learning of Models Pre-Trained on Different Features with Consensus Graphs

Learning an effective global model on private and decentralized datasets...
research
07/19/2022

SphereFed: Hyperspherical Federated Learning

Federated Learning aims at training a global model from multiple decentr...
research
11/29/2021

SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learning

Efficient federated learning is one of the key challenges for training a...
research
07/10/2023

FedYolo: Augmenting Federated Learning with Pretrained Transformers

The growth and diversity of machine learning applications motivate a ret...
research
07/09/2021

Lithography Hotspot Detection via Heterogeneous Federated Learning with Local Adaptation

As technology scaling is approaching the physical limit, lithography hot...
research
03/30/2022

Towards Collaborative Intelligence: Routability Estimation based on Decentralized Private Data

Applying machine learning (ML) in design flow is a popular trend in EDA ...

Please sign up or login with your details

Forgot password? Click here to reset