Hyp-RL : Hyperparameter Optimization by Reinforcement Learning

by   Hadi S. Jomaa, et al.

Hyperparameter tuning is an omnipresent problem in machine learning as it is an integral aspect of obtaining the state-of-the-art performance for any model. Most often, hyperparameters are optimized just by training a model on a grid of possible hyperparameter values and taking the one that performs best on a validation sample (grid search). More recently, methods have been introduced that build a so-called surrogate model that predicts the validation loss for a specific hyperparameter setting, model and dataset and then sequentially select the next hyperparameter to test, based on a heuristic function of the expected value and the uncertainty of the surrogate model called acquisition function (sequential model-based Bayesian optimization, SMBO). In this paper we model the hyperparameter optimization problem as a sequential decision problem, which hyperparameter to test next, and address it with reinforcement learning. This way our model does not have to rely on a heuristic acquisition function like SMBO, but can learn which hyperparameters to test next based on the subsequent reduction in validation loss they will eventually lead to, either because they yield good models themselves or because they allow the hyperparameter selection policy to build a better surrogate model that is able to choose better hyperparameters later on. Experiments on a large battery of 50 data sets demonstrate that our method outperforms the state-of-the-art approaches for hyperparameter learning.


page 1

page 2

page 3

page 4


Improving Hyperparameter Optimization by Planning Ahead

Hyperparameter optimization (HPO) is generally treated as a bi-level opt...

Hyperboost: Hyperparameter Optimization by Gradient Boosting surrogate models

Bayesian Optimization is a popular tool for tuning algorithms in automat...

Hyperparameter Optimization with Differentiable Metafeatures

Metafeatures, or dataset characteristics, have been shown to improve the...

HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shifts

In this work, we propose a hyperparameter optimization method named Hype...

One-Shot Bayes Opt with Probabilistic Population Based Training

Selecting optimal hyperparameters is a key challenge in machine learning...

On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learning

Model-based Reinforcement Learning (MBRL) is a promising framework for l...

Optimizing Hyperparameters with Conformal Quantile Regression

Many state-of-the-art hyperparameter optimization (HPO) algorithms rely ...

Please sign up or login with your details

Forgot password? Click here to reset