Personalized Federated Learning: An Attentive Collaboration Approach

07/07/2020
by   Yutao Huang, et al.
12

For the challenging computational environment of IOT/edge computing, personalized federated learning allows every client to train a strong personalized cloud model by effectively collaborating with the other clients in a privacy-preserving manner. The performance of personalized federated learning is largely determined by the effectiveness of inter-client collaboration. However, when the data is non-IID across all clients, it is challenging to infer the collaboration relationships between clients without knowing their data distributions. In this paper, we propose to tackle this problem by a novel framework named federated attentive message passing (FedAMP) that allows each client to collaboratively train its own personalized cloud model without using a global model. FedAMP implements an attentive collaboration mechanism by iteratively encouraging clients with more similar model parameters to have stronger collaborations. This adaptively discovers the underlying collaboration relationships between clients, which significantly boosts effectiveness of collaboration and leads to the outstanding performance of FedAMP. We establish the convergence of FedAMP for both convex and non-convex models, and further propose a heuristic method that resembles the FedAMP framework to further improve its performance for federated learning with deep neural networks. Extensive experiments demonstrate the superior performance of our methods in handling non-IID data, dirty data and dropped clients.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 8

02/05/2022

A Coalition Formation Game Approach for Personalized Federated Learning

Facing the challenge of statistical diversity in client local data distr...
11/17/2021

Personalized Federated Learning through Local Memorization

Federated learning allows clients to collaboratively learn statistical m...
04/25/2021

FedSup: A Communication-Efficient Federated Learning Fatigue Driving Behaviors Supervision Framework

With the proliferation of edge smart devices and the Internet of Vehicle...
08/18/2021

Learning to Collaborate

In this paper, we focus on effective learning over a collaborative resea...
07/02/2021

On Bridging Generic and Personalized Federated Learning

Federated learning is promising for its ability to collaboratively train...
06/29/2021

Personalized Federated Learning with Gaussian Processes

Federated learning aims to learn a global model that performs well on cl...
03/02/2022

Personalized Federated Learning With Structure

Knowledge sharing and model personalization are two key components to im...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.