Log In Sign Up

Federated Learning on Non-IID Data Silos: An Experimental Study

by   Qinbin Li, et al.

Machine learning services have been emerging in many data-intensive applications, and their effectiveness highly relies on large-volume high-quality training data. However, due to the increasing privacy concerns and data regulations, training data have been increasingly fragmented, forming distributed databases of multiple data silos (e.g., within different organizations and countries). To develop effective machine learning services, there is a must to exploit data from such distributed databases without exchanging the raw data. Recently, federated learning (FL) has been a solution with growing interests, which enables multiple parties to collaboratively train a machine learning model without exchanging their local data. A key and common challenge on distributed databases is the heterogeneity of the data distribution (i.e., non-IID) among the parties. There have been many FL algorithms to address the learning effectiveness under non-IID data settings. However, there lacks an experimental study on systematically understanding their advantages and disadvantages, as previous studies have very rigid data partitioning strategies among parties, which are hardly representative and thorough. In this paper, to help researchers better understand and study the non-IID data setting in federated learning, we propose comprehensive data partitioning strategies to cover the typical non-IID data cases. Moreover, we conduct extensive experiments to evaluate state-of-the-art FL algorithms. We find that non-IID does bring significant challenges in learning accuracy of FL algorithms, and none of the existing state-of-the-art FL algorithms outperforms others in all cases. Our experiments provide insights for future studies of addressing the challenges in data silos.


page 5

page 9


Model-Contrastive Federated Learning

Federated learning enables multiple parties to collaboratively train a m...

Mitigating Bias in Federated Learning

As methods to create discrimination-aware models develop, they focus on ...

IBM Federated Learning: an Enterprise Framework White Paper V0.1

Federated Learning (FL) is an approach to conduct machine learning witho...

Federated Robustness Propagation: Sharing Adversarial Robustness in Federated Learning

Federated learning (FL) emerges as a popular distributed learning schema...

SoK: Training Machine Learning Models over Multiple Sources with Privacy Preservation

Nowadays, gathering high-quality training data from multiple data contro...

Towards Federated Clustering: A Federated Fuzzy c-Means Algorithm (FFCM)

Federated Learning (FL) is a setting where multiple parties with distrib...

Enabling SQL-based Training Data Debugging for Federated Learning

How can we debug a logistical regression model in a federated learning s...

Code Repositories