DeepAI AI Chat
Log In Sign Up

WAFFLE: Weighted Averaging for Personalized Federated Learning

10/13/2021
by   Martin Beaussart, et al.
EPFL
0

In collaborative or federated learning, model personalization can be a very effective strategy to deal with heterogeneous training data across clients. We introduce WAFFLE (Weighted Averaging For Federated LEarning), a personalized collaborative machine learning algorithm based on SCAFFOLD. SCAFFOLD uses stochastic control variates to converge towards a model close to the globally optimal model even in tasks where the distribution of data and labels across clients is highly skewed. In contrast, WAFFLE uses the Euclidean distance between clients' updates to weigh their individual contributions and thus minimize the trained personalized model loss on the specific agent of interest. Through a series of experiments, we compare our proposed new method to two recent personalized federated learning methods, Weight Erosion and APFL, as well as two global learning methods, federated averaging and SCAFFOLD. We evaluate our method using two categories of non-identical client data distributions (concept shift and label skew) on two benchmark image data sets, MNIST and CIFAR10. Our experiments demonstrate the effectiveness of WAFFLE compared with other methods, as it achieves or improves accuracy with faster convergence.

READ FULL TEXT
06/27/2022

An Empirical Study of Personalized Federated Learning

Federated learning is a distributed machine learning approach in which a...
07/20/2021

Precision-Weighted Federated Learning

Federated Learning using the Federated Averaging algorithm has shown gre...
10/25/2021

Optimal Model Averaging: Towards Personalized Collaborative Learning

In federated learning, differences in the data or objectives between the...
06/16/2022

Using adversarial images to improve outcomes of federated learning for non-IID data

One of the important problems in federated learning is how to deal with ...
07/14/2021

IFedAvg: Interpretable Data-Interoperability for Federated Learning

Recently, the ever-growing demand for privacy-oriented machine learning ...
10/14/2019

SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning

Federated learning is a key scenario in modern large-scale machine learn...
09/16/2021

Subspace Learning for Personalized Federated Optimization

As data is generated and stored almost everywhere, learning a model from...