Personalized Federated Learning using Hypernetworks

03/08/2021
by   Aviv Shamsian, et al.
0

Personalized federated learning is tasked with training machine learning models for multiple clients, each with its own data distribution. The goal is to train personalized models in a collaborative way while accounting for data disparities across clients and reducing communication costs. We propose a novel approach to this problem using hypernetworks, termed pFedHN for personalized Federated HyperNetworks. In this approach, a central hypernetwork model is trained to generate a set of models, one model for each client. This architecture provides effective parameter sharing across clients, while maintaining the capacity to generate unique and diverse personal models. Furthermore, since hypernetwork parameters are never transmitted, this approach decouples the communication cost from the trainable model size. We test pFedHN empirically in several personalized federated learning challenges and find that it outperforms previous methods. Finally, since hypernetworks share information across clients we show that pFedHN can generalize better to new clients whose distributions differ from any client observed during training.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

02/18/2022

PerFED-GAN: Personalized Federated Learning via Generative Adversarial Networks

Federated learning is gaining popularity as a distributed machine learni...
10/13/2021

WAFFLE: Weighted Averaging for Personalized Federated Learning

In collaborative or federated learning, model personalization can be a v...
12/11/2021

FedSoft: Soft Clustered Federated Learning with Proximal Local Updating

Traditionally, clustered federated learning groups clients with the same...
07/02/2021

On Bridging Generic and Personalized Federated Learning

Federated learning is promising for its ability to collaboratively train...
06/29/2021

Personalized Federated Learning with Gaussian Processes

Federated learning aims to learn a global model that performs well on cl...
09/17/2021

Comfetch: Federated Learning of Large Networks on Memory-Constrained Clients via Sketching

A popular application of federated learning is using many clients to tra...
07/14/2021

IFedAvg: Interpretable Data-Interoperability for Federated Learning

Recently, the ever-growing demand for privacy-oriented machine learning ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.