DeepAI AI Chat
Log In Sign Up

Mixed-Variable Bayesian Optimization

by   Erik Daxberger, et al.
University of Cambridge

The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete domains have been widely studied. However, few methods exist for mixed-variable domains. In this paper, we introduce MiVaBo, a novel BO algorithm for the efficient optimization of mixed-variable functions that combines a linear surrogate model based on expressive feature representations with Thompson sampling. We propose two methods to optimize its acquisition function, a challenging problem for mixed-variable domains, and we show that MiVaBo can handle complex constraints over the discrete part of the domain that other methods cannot take into account. Moreover, we provide the first convergence analysis of a mixed-variable BO algorithm. Finally, we show that MiVaBo is significantly more sample efficient than state-of-the-art mixed-variable BO algorithms on hyperparameter tuning tasks.


page 13

page 24


Bayesian Optimization For Multi-Objective Mixed-Variable Problems

Optimizing multiple, non-preferential objectives for mixed-variable, exp...

Black-box Mixed-Variable Optimisation using a Surrogate Model that Satisfies Integer Constraints

A challenging problem in both engineering and computer science is that o...

Tree ensemble kernels for Bayesian optimization with known constraints over mixed-feature spaces

Tree ensembles can be well-suited for black-box optimization tasks such ...

Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization

Optimizing expensive-to-evaluate black-box functions of discrete (and po...

A comparison of mixed-variables Bayesian optimization approaches

Most real optimization problems are defined over a mixed search space wh...

Bayesian Optimization with Binary Auxiliary Information

This paper presents novel mixed-type Bayesian optimization (BO) algorith...

Bayesian Optimization of Text Representations

When applying machine learning to problems in NLP, there are many choice...

Code Repositories


Mixed-Variable Bayesian Optimization (

view repo