FedMR: Fedreated Learning via Model Recombination

08/16/2022
by   Ming Hu, et al.
2

As a promising privacy-preserving machine learning method, Federated Learning (FL) enables global model training across clients without compromising their confidential local data. However, existing FL methods suffer from the problem of low inference performance for unevenly distributed data, since most of them rely on Federated Averaging (FedAvg)-based aggregation. By averaging model parameters in a coarse manner, FedAvg eclipses the individual characteristics of local models, which strongly limits the inference capability of FL. Worse still, in each round of FL training, FedAvg dispatches the same initial local models to clients, which can easily result in stuck-at-local-search for optimal global models. To address the above issues, this paper proposes a novel and effective FL paradigm named FedMR (Federating Model Recombination). Unlike conventional FedAvg-based methods, the cloud server of FedMR shuffles each layer of collected local models and recombines them to achieve new models for local training on clients. Due to the fine-grained model recombination and local training in each FL round, FedMR can quickly figure out one globally optimal model for all the clients. Comprehensive experimental results demonstrate that, compared with state-of-the-art FL methods, FedMR can significantly improve the inference accuracy without causing extra communication overhead.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/23/2022

FedCAT: Towards Accurate Federated Learning via Device Concatenation

As a promising distributed machine learning paradigm, Federated Learning...
research
03/10/2023

Complement Sparsification: Low-Overhead Model Pruning for Federated Learning

Federated Learning (FL) is a privacy-preserving distributed deep learnin...
research
10/28/2022

Federated Learning with Intermediate Representation Regularization

In contrast to centralized model training that involves data collection,...
research
05/26/2020

Continual Local Training for Better Initialization of Federated Models

Federated learning (FL) refers to the learning paradigm that trains mach...
research
06/08/2022

Dap-FL: Federated Learning flourishes by adaptive tuning and secure aggregation

Federated learning (FL), an attractive and promising distributed machine...
research
01/28/2023

CyclicFL: A Cyclic Model Pre-Training Approach to Efficient Federated Learning

Since random initial models in Federated Learning (FL) can easily result...
research
02/03/2021

A Bayesian Federated Learning Framework with Multivariate Gaussian Product

Federated learning (FL) allows multiple clients to collaboratively learn...

Please sign up or login with your details

Forgot password? Click here to reset