On Bridging Generic and Personalized Federated Learning

07/02/2021
by   Hong-You Chen, et al.
0

Federated learning is promising for its ability to collaboratively train models with multiple clients without accessing their data, but vulnerable when clients' data distributions diverge from each other. This divergence further leads to a dilemma: "Should we prioritize the learned model's generic performance (for future use at the server) or its personalized performance (for each client)?" These two, seemingly competing goals have divided the community to focus on one or the other, yet in this paper we show that it is possible to approach both at the same time. Concretely, we propose a novel federated learning framework that explicitly decouples a model's dual duties with two prediction tasks. On the one hand, we introduce a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them. On the other hand, we formulate the personalized predictor as a lightweight adaptive module that is learned to minimize each client's empirical risk on top of the generic predictor. With this two-loss, two-predictor framework which we name Federated Robust Decoupling Fed-RoD, the learned model can simultaneously achieve state-of-the-art generic and personalized performance, essentially bridging the two tasks.

READ FULL TEXT

Authors

page 24

03/08/2021

Personalized Federated Learning using Hypernetworks

Personalized federated learning is tasked with training machine learning...
11/17/2021

Personalized Federated Learning through Local Memorization

Federated learning allows clients to collaboratively learn statistical m...
07/07/2020

Personalized Federated Learning: An Attentive Collaboration Approach

For the challenging computational environment of IOT/edge computing, per...
01/14/2021

Auto-weighted Robust Federated Learning with Corrupted Data Sources

Federated learning provides a communication-efficient and privacy-preser...
02/05/2022

A Coalition Formation Game Approach for Personalized Federated Learning

Facing the challenge of statistical diversity in client local data distr...
11/04/2021

Parameterized Knowledge Transfer for Personalized Federated Learning

In recent years, personalized federated learning (pFL) has attracted inc...
05/09/2022

Layer-wised Model Aggregation for Personalized Federated Learning

Personalized Federated Learning (pFL) not only can capture the common pr...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.