Bayesian Optimization Using Monotonicity Information and Its Application in Machine Learning Hyperparameter

02/10/2018
by   Wenyi Wang, et al.
0

We propose an algorithm for a family of optimization problems where the objective can be decomposed as a sum of functions with monotonicity properties. The motivating problem is optimization of hyperparameters of machine learning algorithms, where we argue that the objective, validation error, can be decomposed as monotonic functions of the hyperparameters. Our proposed algorithm adapts Bayesian optimization methods to incorporate the monotonicity constraints. We illustrate the advantages of exploiting monotonicity using illustrative examples and demonstrate the improvements in optimization efficiency for some machine learning hyperparameter tuning applications.

READ FULL TEXT
research
03/21/2020

Towards Automatic Bayesian Optimization: A first step involving acquisition functions

Bayesian Optimization is the state of the art technique for the optimiza...
research
08/19/2019

Towards Assessing the Impact of Bayesian Optimization's Own Hyperparameters

Bayesian Optimization (BO) is a common approach for hyperparameter optim...
research
09/20/2021

SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization

Algorithm parameters, in particular hyperparameters of machine learning ...
research
02/06/2019

Fast Hyperparameter Tuning using Bayesian Optimization with Directional Derivatives

In this paper we develop a Bayesian optimization based hyperparameter tu...
research
07/13/2021

Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges

Most machine learning algorithms are configured by one or several hyperp...
research
09/08/2020

Hyperparameter Optimization via Sequential Uniform Designs

Hyperparameter tuning or optimization plays a central role in the automa...
research
09/20/2019

Bayesian Optimization for Iterative Learning

The success of deep (reinforcement) learning systems crucially depends o...

Please sign up or login with your details

Forgot password? Click here to reset