Behavior Mimics Distribution: Combining Individual and Group Behaviors for Federated Learning

06/23/2021
by   Hua Huang, et al.
0

Federated Learning (FL) has become an active and promising distributed machine learning paradigm. As a result of statistical heterogeneity, recent studies clearly show that the performance of popular FL methods (e.g., FedAvg) deteriorates dramatically due to the client drift caused by local updates. This paper proposes a novel Federated Learning algorithm (called IGFL), which leverages both Individual and Group behaviors to mimic distribution, thereby improving the ability to deal with heterogeneity. Unlike existing FL methods, our IGFL can be applied to both client and server optimization. As a by-product, we propose a new attention-based federated learning in the server optimization of IGFL. To the best of our knowledge, this is the first time to incorporate attention mechanisms into federated optimization. We conduct extensive experiments and show that IGFL can significantly improve the performance of existing federated learning methods. Especially when the distributions of data among individuals are diverse, IGFL can improve the classification accuracy by about 13

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

11/23/2020

LINDT: Tackling Negative Federated Learning with Local Adaptation

Federated Learning (FL) is a promising distributed learning paradigm, wh...
02/29/2020

Adaptive Federated Optimization

Federated learning is a distributed machine learning paradigm in which a...
11/23/2020

Improving Federated Relational Data Modeling via Basis Alignment and Weight Penalty

Federated learning (FL) has attracted increasing attention in recent yea...
12/20/2021

HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images

Multiple medical institutions collaboratively training a model using fed...
12/03/2020

Federated Learning with Diversified Preference for Humor Recognition

Understanding humor is critical to creative language modeling with many ...
05/19/2021

Prototype Guided Federated Learning of Visual Feature Representations

Federated Learning (FL) is a framework which enables distributed model t...
10/15/2021

FedSLD: Federated Learning with Shared Label Distribution for Medical Image Classification

Machine learning in medical research, by nature, needs careful attention...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.