Federated Learning with Non-IID Data

by   Yue Zhao, et al.

Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55 non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the edge devices. Experiments show that accuracy can be increased by 30 CIFAR-10 dataset with only 5



There are no comments yet.


page 1

page 2

page 3

page 4


FedCD: Improving Performance in non-IID Federated Learning

Federated learning has been widely applied to enable decentralized devic...

Privacy is What We Care About: Experimental Investigation of Federated Learning on Edge Devices

Federated Learning enables training of a general model through edge devi...

WAFFLe: Weight Anonymized Factorization for Federated Learning

In domains where data are sensitive or private, there is great value in ...

One-Shot Federated Learning with Neuromorphic Processors

Being very low power, the use of neuromorphic processors in mobile devic...

An On-Device Federated Learning Approach for Cooperative Anomaly Detection

Most edge AI focuses on prediction tasks on resource-limited edge device...

FedZKT: Zero-Shot Knowledge Transfer towards Heterogeneous On-Device Models in Federated Learning

Federated learning enables distributed devices to collaboratively learn ...

Weight Divergence Driven Divide-and-Conquer Approach for Optimal Federated Learning from non-IID Data

Federated Learning allows training of data stored in distributed devices...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.