Log In Sign Up

Adaptive Local Bayesian Optimization Over Multiple Discrete Variables

by   Taehyeon Kim, et al.

In the machine learning algorithms, the choice of the hyperparameter is often an art more than a science, requiring labor-intensive search with expert experience. Therefore, automation on hyperparameter optimization to exclude human intervention is a great appeal, especially for the black-box functions. Recently, there have been increasing demands of solving such concealed tasks for better generalization, though the task-dependent issue is not easy to solve. The Black-Box Optimization challenge (NeurIPS 2020) required competitors to build a robust black-box optimizer across different domains of standard machine learning problems. This paper describes the approach of team KAIST OSI in a step-wise manner, which outperforms the baseline algorithms by up to +20.39 region reliability. Then, we design a combinatorial kernel for a Gaussian process kernel. In a similar vein, we combine the methodology of Bayesian and multi-armed bandit,(MAB) approach to select the values with the consideration of the variable types; the real and integer variables are with Bayesian, while the boolean and categorical variables are with MAB. Empirical evaluations demonstrate that our method outperforms the existing methods across different tasks.


page 1

page 2

page 3

page 4


Enhancing Explainability of Hyperparameter Optimization via Bayesian Algorithm Execution

Despite all the benefits of automated hyperparameter optimization (HPO),...

Stepwise Model Selection for Sequence Prediction via Deep Kernel Learning

An essential problem in automated machine learning (AutoML) is that of m...

Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques

Designers reportedly struggle with design optimization tasks where they ...

Naive Automated Machine Learning – A Late Baseline for AutoML

Automated Machine Learning (AutoML) is the problem of automatically find...

Warm Starting CMA-ES for Hyperparameter Optimization

Hyperparameter optimization (HPO), formulated as black-box optimization ...

Bayesian Optimisation over Multiple Continuous and Categorical Inputs

Efficient optimisation of black-box problems that comprise both continuo...