DeepAI AI Chat
Log In Sign Up

Mixed-Variable Bayesian Optimization

07/02/2019
by   Erik Daxberger, et al.
University of Cambridge
12

The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete domains have been widely studied. However, few methods exist for mixed-variable domains. In this paper, we introduce MiVaBo, a novel BO algorithm for the efficient optimization of mixed-variable functions that combines a linear surrogate model based on expressive feature representations with Thompson sampling. We propose two methods to optimize its acquisition function, a challenging problem for mixed-variable domains, and we show that MiVaBo can handle complex constraints over the discrete part of the domain that other methods cannot take into account. Moreover, we provide the first convergence analysis of a mixed-variable BO algorithm. Finally, we show that MiVaBo is significantly more sample efficient than state-of-the-art mixed-variable BO algorithms on hyperparameter tuning tasks.

READ FULL TEXT

page 13

page 24

01/30/2022

Bayesian Optimization For Multi-Objective Mixed-Variable Problems

Optimizing multiple, non-preferential objectives for mixed-variable, exp...
06/08/2020

Black-box Mixed-Variable Optimisation using a Surrogate Model that Satisfies Integer Constraints

A challenging problem in both engineering and computer science is that o...
07/02/2022

Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces

Tree ensembles can be well-suited for black-box optimization tasks such ...
10/18/2022

Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization

Optimizing expensive-to-evaluate black-box functions of discrete (and po...
10/30/2021

A comparison of mixed-variables Bayesian optimization approaches

Most real optimization problems are defined over a mixed search space wh...
06/17/2019

Bayesian Optimization with Binary Auxiliary Information

This paper presents novel mixed-type Bayesian optimization (BO) algorith...
03/02/2015

Bayesian Optimization of Text Representations

When applying machine learning to problems in NLP, there are many choice...

Code Repositories

mixed_variable_bo

Mixed-Variable Bayesian Optimization (https://arxiv.org/abs/1907.01329)


view repo