Federated Learning with Non-IID Data

06/02/2018
by   Yue Zhao, et al.
0

Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55 non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the edge devices. Experiments show that accuracy can be increased by 30 CIFAR-10 dataset with only 5

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/17/2020

FedCD: Improving Performance in non-IID Federated Learning

Federated learning has been widely applied to enable decentralized devic...
research
11/11/2019

Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devices

Federated Learning enables training of a general model through edge devi...
research
08/13/2020

WAFFLe: Weight Anonymized Factorization for Federated Learning

In domains where data are sensitive or private, there is great value in ...
research
02/27/2020

An On-Device Federated Learning Approach for Cooperative Anomaly Detection

Most edge AI focuses on prediction tasks on resource-limited edge device...
research
09/08/2021

FedZKT: Zero-Shot Knowledge Transfer towards Heterogeneous On-Device Models in Federated Learning

Federated learning enables distributed devices to collaboratively learn ...
research
06/28/2021

Weight Divergence Driven Divide-and-Conquer Approach for Optimal Federated Learning from non-IID Data

Federated Learning allows training of data stored in distributed devices...
research
01/14/2023

FedSSC: Shared Supervised-Contrastive Federated Learning

Federated learning is widely used to perform decentralized training of a...

Please sign up or login with your details

Forgot password? Click here to reset