Exploiting Reuse in Pipeline-Aware Hyperparameter Tuning

03/12/2019
by   Liam Li, et al.
18

Hyperparameter tuning of multi-stage pipelines introduces a significant computational burden. Motivated by the observation that work can be reused across pipelines if the intermediate computations are the same, we propose a pipeline-aware approach to hyperparameter tuning. Our approach optimizes both the design and execution of pipelines to maximize reuse. We design pipelines amenable for reuse by (i) introducing a novel hybrid hyperparameter tuning method called gridded random search, and (ii) reducing the average training time in pipelines by adapting early-stopping hyperparameter tuning approaches. We then realize the potential for reuse during execution by introducing a novel caching problem for ML workloads which we pose as a mixed integer linear program (ILP), and subsequently evaluating various caching heuristics relative to the optimal solution of the ILP. We conduct experiments on simulated and real-world machine learning pipelines to show that a pipeline-aware approach to hyperparameter tuning can offer over an order-of-magnitude speedup over independently evaluating pipeline configurations.

READ FULL TEXT
research
01/26/2021

Incremental Search Space Construction for Machine Learning Pipeline Synthesis

Automated machine learning (AutoML) aims for constructing machine learni...
research
04/01/2019

Adaptive Bayesian Linear Regression for Automated Machine Learning

To solve a machine learning problem, one typically needs to perform data...
research
06/26/2021

Automated Evolutionary Approach for the Design of Composite Machine Learning Pipelines

The effectiveness of the machine learning methods for real-world tasks d...
research
10/13/2018

Massively Parallel Hyperparameter Tuning

Modern learning models are characterized by large hyperparameter spaces....
research
10/23/2018

Preprocessor Selection for Machine Learning Pipelines

Much of the work in metalearning has focused on classifier selection, co...
research
08/04/2022

ACE: Adaptive Constraint-aware Early Stopping in Hyperparameter Optimization

Deploying machine learning models requires high model quality and needs ...
research
02/09/2023

Hyperparameter Search Is All You Need For Training-Agnostic Backdoor Robustness

Commoditization and broad adoption of machine learning (ML) technologies...

Please sign up or login with your details

Forgot password? Click here to reset