Single-shot Hyper-parameter Optimization for Federated Learning: A General Algorithm Analysis

02/16/2022
by   Yi Zhou, et al.
0

We address the relatively unexplored problem of hyper-parameter optimization (HPO) for federated learning (FL-HPO). We introduce Federated Loss SuRface Aggregation (FLoRA), a general FL-HPO solution framework that can address use cases of tabular data and any Machine Learning (ML) model including gradient boosting training algorithms and therefore further expands the scope of FL-HPO. FLoRA enables single-shot FL-HPO: identifying a single set of good hyper-parameters that are subsequently used in a single FL training. Thus, it enables FL-HPO solutions with minimal additional communication overhead compared to FL training without HPO. We theoretically characterize the optimality gap of FL-HPO, which explicitly accounts for the heterogeneous non-IID nature of the parties' local data distributions, a dominant characteristic of FL systems. Our empirical evaluation of FLoRA for multiple ML algorithms on seven OpenML datasets demonstrates significant model accuracy improvements over the considered baseline, and robustness to increasing number of parties involved in FL-HPO training.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/15/2021

FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning

We address the relatively unexplored problem of hyper-parameter optimiza...
research
07/15/2021

Genetic CFL: Optimization of Hyper-Parameters in Clustered Federated Learning

Federated learning (FL) is a distributed model for deep learning that in...
research
03/23/2022

Adaptive Aggregation For Federated Learning

Advances in federated learning (FL) algorithms,along with technologies l...
research
12/14/2021

Federated Nearest Neighbor Classification with a Colony of Fruit-Flies: With Supplement

The mathematical formalization of a neurological mechanism in the olfact...
research
07/15/2022

Communication-Efficient Diffusion Strategy for Performance Improvement of Federated Learning with Non-IID Data

Federated learning (FL) is a novel learning paradigm that addresses the ...
research
02/25/2022

Towards an Accountable and Reproducible Federated Learning: A FactSheets Approach

Federated Learning (FL) is a novel paradigm for the shared training of m...
research
08/07/2023

FLIPS: Federated Learning using Intelligent Participant Selection

This paper presents the design and implementation of FLIPS, a middleware...

Please sign up or login with your details

Forgot password? Click here to reset