Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings

03/03/2023
by   Aryan Deshwal, et al.
0

We consider the problem of optimizing expensive black-box functions over high-dimensional combinatorial spaces which arises in many science, engineering, and ML applications. We use Bayesian Optimization (BO) and propose a novel surrogate modeling approach for efficiently handling a large number of binary and categorical parameters. The key idea is to select a number of discrete structures from the input space (the dictionary) and use them to define an ordinal embedding for high-dimensional combinatorial structures. This allows us to use existing Gaussian process models for continuous spaces. We develop a principled approach based on binary wavelets to construct dictionaries for binary spaces, and propose a randomized construction method that generalizes to categorical spaces. We provide theoretical justification to support the effectiveness of the dictionary-based embeddings. Our experiments on diverse real-world benchmarks demonstrate the effectiveness of our proposed surrogate modeling approach over state-of-the-art BO methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/01/2021

Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces

We consider the problem of optimizing combinatorial spaces (e.g., sequen...
research
12/14/2020

Mercer Features for Efficient Combinatorial Bayesian Optimization

Bayesian optimization (BO) is an efficient framework for solving black-b...
research
07/02/2023

Bounce: a Reliable Bayesian Optimization Algorithm for Combinatorial and Mixed Spaces

Impactful applications such as materials discovery, hardware design, neu...
research
11/03/2020

Bayesian Variational Optimization for Combinatorial Spaces

This paper focuses on Bayesian Optimization in combinatorial spaces. In ...
research
08/18/2020

Scalable Combinatorial Bayesian Optimization with Tractable Statistical models

We study the problem of optimizing expensive blackbox functions over com...
research
06/08/2021

Bayesian Optimization over Hybrid Spaces

We consider the problem of optimizing hybrid structures (mixture of disc...
research
10/21/2019

Integrals over Gaussians under Linear Domain Constraints

Integrals of linearly constrained multivariate Gaussian densities are a ...

Please sign up or login with your details

Forgot password? Click here to reset