UniFed: A Benchmark for Federated Learning Frameworks

07/21/2022
by   Xiaoyuan Liu, et al.
11

Federated Learning (FL) has become a practical and popular paradigm in machine learning. However, currently, there is no systematic solution that covers diverse use cases. Practitioners often face the challenge of how to select a matching FL framework for their use case. In this work, we present UniFed, the first unified benchmark for standardized evaluation of the existing open-source FL frameworks. With 15 evaluation scenarios, we present both qualitative and quantitative evaluation results of nine existing popular open-sourced FL frameworks, from the perspectives of functionality, usability, and system performance. We also provide suggestions on framework selection based on the benchmark conclusions and point out future improvement directions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/09/2023

An Empirical Study of Bugs in Open-Source Federated Learning Framework

Federated learning (FL), as a decentralized machine learning solution to...
research
03/08/2023

Model-Agnostic Federated Learning

Since its debut in 2016, Federated Learning (FL) has been tied to the in...
research
07/05/2023

Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives

Traditional Federated Learning (FL) follows a server-domincated cooperat...
research
06/08/2022

FedHPO-B: A Benchmark Suite for Federated Hyperparameter Optimization

Hyperparameter optimization (HPO) is crucial for machine learning algori...
research
04/04/2023

SLPerf: a Unified Framework for Benchmarking Split Learning

Data privacy concerns has made centralized training of data, which is sc...
research
07/25/2022

AMLB: an AutoML Benchmark

Comparing different AutoML frameworks is notoriously challenging and oft...
research
07/28/2023

The Applicability of Federated Learning to Official Statistics

This work investigates the potential of Federated Learning (FL) for offi...

Please sign up or login with your details

Forgot password? Click here to reset