A Principled Approach to Data Valuation for Federated Learning

09/14/2020
by   Tianhao Wang, et al.
25

Federated learning (FL) is a popular technique to train machine learning (ML) models on decentralized data sources. In order to sustain long-term participation of data owners, it is important to fairly appraise each data source and compensate data owners for their contribution to the training process. The Shapley value (SV) defines a unique payoff scheme that satisfies many desiderata for a data value notion. It has been increasingly used for valuing training data in centralized learning. However, computing the SV requires exhaustively evaluating the model performance on every subset of data sources, which incurs prohibitive communication cost in the federated setting. Besides, the canonical SV ignores the order of data sources during training, which conflicts with the sequential nature of FL. This paper proposes a variant of the SV amenable to FL, which we call the federated Shapley value. The federated SV preserves the desirable properties of the canonical SV while it can be calculated without incurring extra communication cost and is also able to capture the effect of participation order on data value. We conduct a thorough empirical study of the federated SV on a range of tasks, including noisy label detection, adversarial participant detection, and data summarization on different benchmark datasets, and demonstrate that it can reflect the real utility of data sources for FL and has the potential to enhance system robustness, security, and efficiency. We also report and analyze "failure cases" and hope to stimulate future research.

READ FULL TEXT

page 1

page 2

page 3

page 4

01/07/2022

Fair and efficient contribution valuation for vertical federated learning

Federated learning is a popular technology for training machine learning...
09/15/2022

How Much Does It Cost to Train a Machine Learning Model over Distributed Data Sources?

Federated learning (FL) is one of the most appealing alternatives to the...
02/27/2019

Towards Efficient Data Valuation Based on the Shapley Value

"How much is my data worth?" is an increasingly common question posed by...
09/05/2022

Boost Decentralized Federated Learning in Vehicular Networks by Diversifying Data Sources

Recently, federated learning (FL) has received intensive research becaus...
01/25/2021

Failure Prediction in Production Line Based on Federated Learning: An Empirical Study

Data protection across organizations is limiting the application of cent...
11/30/2018

LoAdaBoost:Loss-Based AdaBoost Federated Machine Learning on medical Data

Medical data are valuable for improvement of health care, policy making ...
08/07/2021

Secure Neuroimaging Analysis using Federated Learning with Homomorphic Encryption

Federated learning (FL) enables distributed computation of machine learn...