Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity

05/02/2021
by   Saeed Vahidian, et al.
0

The traditional approach in FL tries to learn a single global model collaboratively with the help of many clients under the orchestration of a central server. However, learning a single global model might not work well for all clients participating in the FL under data heterogeneity. Therefore, the personalization of the global model becomes crucial in handling the challenges that arise with statistical heterogeneity and the non-IID distribution of data. Unlike prior works, in this work we propose a new approach for obtaining a personalized model from a client-level objective. This further motivates all clients to participate in federation even under statistical heterogeneity in order to improve their performance, instead of merely being a source of data and model training for the central server. To realize this personalization, we leverage finding a small subnetwork for each client by applying hybrid pruning (combination of structured and unstructured pruning), and unstructured pruning. Through a range of experiments on different benchmarks, we observed that the clients with similar data (labels) share similar personal parameters. By finding a subnetwork for each client ...

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/16/2020

Personalized Federated Learning with Moreau Envelopes

Federated learning (FL) is a decentralized and privacy-preserving machin...
11/16/2021

Inference-Time Personalized Federated Learning

In Federated learning (FL), multiple clients collaborate to learn a mode...
07/28/2021

New Metrics to Evaluate the Performance and Fairness of Personalized Federated Learning

In Federated Learning (FL), the clients learn a single global model (Fed...
07/29/2021

QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning

Traditionally, federated learning (FL) aims to train a single global mod...
05/31/2021

Unifying Distillation with Personalization in Federated Learning

Federated learning (FL) is a decentralized privacy-preserving learning t...
04/22/2022

A Closer Look at Personalization in Federated Image Classification

Federated Learning (FL) is developed to learn a single global model acro...
05/11/2021

Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Data

We propose FEDENHANCE, an unsupervised federated learning (FL) approach ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.