Federated Learning with Non-IID Data

06/02/2018
by   Yue Zhao, et al.
0

Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55 non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the edge devices. Experiments show that accuracy can be increased by 30 CIFAR-10 dataset with only 5

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/17/2020

FedCD: Improving Performance in non-IID Federated Learning

Federated learning has been widely applied to enable decentralized devic...
11/11/2019

Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devices

Federated Learning enables training of a general model through edge devi...
08/13/2020

WAFFLe: Weight Anonymized Factorization for Federated Learning

In domains where data are sensitive or private, there is great value in ...
11/01/2020

One-Shot Federated Learning with Neuromorphic Processors

Being very low power, the use of neuromorphic processors in mobile devic...
02/27/2020

An On-Device Federated Learning Approach for Cooperative Anomaly Detection

Most edge AI focuses on prediction tasks on resource-limited edge device...
09/08/2021

FedZKT: Zero-Shot Knowledge Transfer towards Heterogeneous On-Device Models in Federated Learning

Federated learning enables distributed devices to collaboratively learn ...
06/28/2021

Weight Divergence Driven Divide-and-Conquer Approach for Optimal Federated Learning from non-IID Data

Federated Learning allows training of data stored in distributed devices...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.