A Principled Approach to Data Valuation for Federated Learning

09/14/2020
by   Tianhao Wang, et al.
25

Federated learning (FL) is a popular technique to train machine learning (ML) models on decentralized data sources. In order to sustain long-term participation of data owners, it is important to fairly appraise each data source and compensate data owners for their contribution to the training process. The Shapley value (SV) defines a unique payoff scheme that satisfies many desiderata for a data value notion. It has been increasingly used for valuing training data in centralized learning. However, computing the SV requires exhaustively evaluating the model performance on every subset of data sources, which incurs prohibitive communication cost in the federated setting. Besides, the canonical SV ignores the order of data sources during training, which conflicts with the sequential nature of FL. This paper proposes a variant of the SV amenable to FL, which we call the federated Shapley value. The federated SV preserves the desirable properties of the canonical SV while it can be calculated without incurring extra communication cost and is also able to capture the effect of participation order on data value. We conduct a thorough empirical study of the federated SV on a range of tasks, including noisy label detection, adversarial participant detection, and data summarization on different benchmark datasets, and demonstrate that it can reflect the real utility of data sources for FL and has the potential to enhance system robustness, security, and efficiency. We also report and analyze "failure cases" and hope to stimulate future research.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/07/2022

Fair and efficient contribution valuation for vertical federated learning

Federated learning is a popular technology for training machine learning...
research
08/21/2023

Federated Learning for Connected and Automated Vehicles: A Survey of Existing Approaches and Challenges

Machine learning (ML) is widely used for key tasks in Connected and Auto...
research
09/15/2022

How Much Does It Cost to Train a Machine Learning Model over Distributed Data Sources?

Federated learning (FL) is one of the most appealing alternatives to the...
research
02/27/2019

Towards Efficient Data Valuation Based on the Shapley Value

"How much is my data worth?" is an increasingly common question posed by...
research
09/05/2022

Boost Decentralized Federated Learning in Vehicular Networks by Diversifying Data Sources

Recently, federated learning (FL) has received intensive research becaus...
research
06/07/2023

Phoenix: A Federated Generative Diffusion Model

Generative AI has made impressive strides in enabling users to create di...
research
11/30/2018

LoAdaBoost:Loss-Based AdaBoost Federated Machine Learning on medical Data

Medical data are valuable for improvement of health care, policy making ...

Please sign up or login with your details

Forgot password? Click here to reset