Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?

01/30/2023
by   Juliusz Ziomek, et al.
0

Learning decompositions of expensive-to-evaluate black-box functions promises to scale Bayesian optimisation (BO) to high-dimensional problems. However, the success of these techniques depends on finding proper decompositions that accurately represent the black-box. While previous works learn those decompositions based on data, we investigate data-independent decomposition sampling rules in this paper. We find that data-driven learners of decompositions can be easily misled towards local decompositions that do not hold globally across the search space. Then, we formally show that a random tree-based decomposition sampler exhibits favourable theoretical guarantees that effectively trade off maximal information gain and functional mismatch between the actual black-box and its surrogate as provided by the decomposition. Those results motivate the development of the random decomposition upper-confidence bound algorithm (RDUCB) that is straightforward to implement - (almost) plug-and-play - and, surprisingly, yields significant empirical gains compared to the previous state-of-the-art on a comprehensive set of benchmarks. We also confirm the plug-and-play nature of our modelling component by integrating our method with HEBO, showing improved practical gains in the highest dimensional tasks from Bayesmark.

READ FULL TEXT
research
01/06/2023

On the coalitional decomposition of parameters of interest

Understanding the behavior of a black-box model with probabilistic input...
research
07/17/2018

Faster Algorithms via Waring Decompositions

We show that decompositions of certain polynomials as sums of powers of ...
research
07/01/2020

Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search

High dimensional black-box optimization has broad applications but remai...
research
01/28/2020

An operational information decomposition via synergistic disclosure

Multivariate information decompositions hold promise to yield insight in...
research
11/27/2016

Embedded Bandits for Large-Scale Black-Box Optimization

Random embedding has been applied with empirical success to large-scale ...
research
06/07/2021

High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning

We introduce a method based on deep metric learning to perform Bayesian ...
research
04/13/2022

Chaining of Numerical Black-box Algorithms: Warm-Starting and Switching Points

Dynamic algorithm selection can be beneficial for solving numerical blac...

Please sign up or login with your details

Forgot password? Click here to reset