DeepAI AI Chat
Log In Sign Up

On Bridging Generic and Personalized Federated Learning

07/02/2021
by   Hong-You Chen, et al.
0

Federated learning is promising for its ability to collaboratively train models with multiple clients without accessing their data, but vulnerable when clients' data distributions diverge from each other. This divergence further leads to a dilemma: "Should we prioritize the learned model's generic performance (for future use at the server) or its personalized performance (for each client)?" These two, seemingly competing goals have divided the community to focus on one or the other, yet in this paper we show that it is possible to approach both at the same time. Concretely, we propose a novel federated learning framework that explicitly decouples a model's dual duties with two prediction tasks. On the one hand, we introduce a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them. On the other hand, we formulate the personalized predictor as a lightweight adaptive module that is learned to minimize each client's empirical risk on top of the generic predictor. With this two-loss, two-predictor framework which we name Federated Robust Decoupling Fed-RoD, the learned model can simultaneously achieve state-of-the-art generic and personalized performance, essentially bridging the two tasks.

READ FULL TEXT
03/08/2021

Personalized Federated Learning using Hypernetworks

Personalized federated learning is tasked with training machine learning...
07/07/2020

Personalized Federated Learning: An Attentive Collaboration Approach

For the challenging computational environment of IOT/edge computing, per...
12/31/2020

PFL-MoE: Personalized Federated Learning Based on Mixture of Experts

Federated learning (FL) is an emerging distributed machine learning para...
01/14/2021

Auto-weighted Robust Federated Learning with Corrupted Data Sources

Federated learning provides a communication-efficient and privacy-preser...
09/15/2023

Intent Detection at Scale: Tuning a Generic Model using Relevant Intents

Accurately predicting the intent of customer support requests is vital f...
04/16/2023

Federated Learning of Shareable Bases for Personalization-Friendly Image Classification

Personalized federated learning (PFL) aims to harness the collective wis...
08/16/2023

Towards Personalized Federated Learning via Heterogeneous Model Reassembly

This paper focuses on addressing the practical yet challenging problem o...