Bayesian Optimistic Optimisation with Exponentially Decaying Regret

by   Hung Tran-The, et al.

Bayesian optimisation (BO) is a well-known efficient algorithm for finding the global optimum of expensive, black-box functions. The current practical BO algorithms have regret bounds ranging from π’ͺ(logN/√(N)) to π’ͺ(e^-√(N)), where N is the number of evaluations. This paper explores the possibility of improving the regret bound in the noiseless setting by intertwining concepts from BO and tree-based optimistic optimisation which are based on partitioning the search space. We propose the BOO algorithm, a first practical approach which can achieve an exponential regret bound with order π’ͺ(N^-√(N)) under the assumption that the objective function is sampled from a Gaussian process with a MatΓ©rn kernel with smoothness parameter Ξ½ > 4 +D/2, where D is the number of dimensions. We perform experiments on optimisation of various synthetic functions and machine learning hyperparameter tuning tasks and show that our algorithm outperforms baselines.


page 1

page 2

page 3

page 4

βˆ™ 09/05/2020

Sub-linear Regret Bounds for Bayesian Optimisation in Unknown Search Spaces

Bayesian optimisation is a popular method for efficient optimisation of ...
βˆ™ 03/27/2020

Incorporating Expert Prior in Bayesian Optimisation via Space Warping

Bayesian optimisation is a well-known sample-efficient method for the op...
βˆ™ 12/05/2019

Ordinal Bayesian Optimisation

Bayesian optimisation is a powerful tool to solve expensive black-box pr...
βˆ™ 02/01/2023

Delayed Feedback in Kernel Bandits

Black box optimisation of an unknown function from expensive and noisy e...
βˆ™ 09/08/2020

Sequential Subspace Search for Functional Bayesian Optimization Incorporating Experimenter Intuition

We propose an algorithm for Bayesian functional optimisation - that is, ...
βˆ™ 05/25/2018

Parallel Architecture and Hyperparameter Search via Successive Halving and Classification

We present a simple and powerful algorithm for parallel black box optimi...
βˆ™ 05/20/2022

Machine Learning for Combinatorial Optimisation of Partially-Specified Problems: Regret Minimisation as a Unifying Lens

It is increasingly common to solve combinatorial optimisation problems t...

Please sign up or login with your details

Forgot password? Click here to reset