FedCCEA : A Practical Approach of Client Contribution Evaluation for Federated Learning

06/04/2021
by   Sung Kuk Shyn, et al.
0

Client contribution evaluation, also known as data valuation, is a crucial approach in federated learning(FL) for client selection and incentive allocation. However, due to restrictions of accessibility of raw data, only limited information such as local weights and local data size of each client is open for quantifying the client contribution. Using data size from available information, we introduce an empirical evaluation method called Federated Client Contribution Evaluation through Accuracy Approximation(FedCCEA). This method builds the Accuracy Approximation Model(AAM), which estimates a simulated test accuracy using inputs of sampled data size and extracts the clients' data quality and data size to measure client contribution. FedCCEA strengthens some advantages: (1) enablement of data size selection to the clients, (2) feasible evaluation time regardless of the number of clients, and (3) precise estimation in non-IID settings. We demonstrate the superiority of FedCCEA compared to previous methods through several experiments: client contribution distribution, client removal, and robustness test to partial participation.

READ FULL TEXT

Authors

page 15

07/09/2020

Client Adaptation improves Federated Learning with Simulated Non-IID Clients

We present a federated learning approach for learning a client adaptable...
12/15/2020

Personalized Federated Learning with First Order Model Optimization

While federated learning traditionally aims to train a single global mod...
02/18/2022

Incentive Mechanism Design for Joint Resource Allocation in Blockchain-based Federated Learning

Blockchain-based federated learning (BCFL) has recently gained tremendou...
05/03/2022

FedRN: Exploiting k-Reliable Neighbors Towards Robust Federated Learning

Robustness is becoming another important challenge of federated learning...
02/26/2021

Efficient Client Contribution Evaluation for Horizontal Federated Learning

In federated learning (FL), fair and accurate measurement of the contrib...
08/20/2019

Towards Effective Device-Aware Federated Learning

With the wealth of information produced by social networks, smartphones,...
06/17/2022

Decentralized adaptive clustering of deep nets is beneficial for client collaboration

We study the problem of training personalized deep learning models in a ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.