Sparse Spectrum Gaussian Process for Bayesian Optimisation

06/21/2019
by   Ang Yang, et al.
0

We propose a novel sparse spectrum approximation of Gaussian process (GP) tailored for Bayesian optimisation. Whilst the current sparse spectrum methods provide good approximations for regression problems, it is observed that this particular form of sparse approximations generates an overconfident GP, i.e. it predicts less variance than the original GP. Since the balance between predictive mean and the predictive variance is a key determinant in the success of Bayesian optimisation, the current sparse spectrum methods are less suitable. We derive a regularised marginal likelihood for finding the optimal frequencies in optimisation problems. The regulariser trades the accuracy in the model fitting with the targeted increase in the variance of the resultant GP. We first consider the entropy of the distribution over the maxima as the regulariser that needs to be maximised. Later we show that the Expected Improvement acquisition function can also be used as a proxy for that, thus making the optimisation less computationally expensive. Experiments show an increase in the Bayesian optimisation convergence rate over the vanilla sparse spectrum method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/08/2020

Randomised Gaussian Process Upper Confidence Bound for Bayesian Optimisation

In order to improve the performance of Bayesian optimisation, we develop...
research
11/18/2016

A Generalized Stochastic Variational Bayesian Hyperparameter Learning Framework for Sparse Spectrum Gaussian Process Regression

While much research effort has been dedicated to scaling up sparse Gauss...
research
10/17/2019

Achieving Robustness to Aleatoric Uncertainty with Heteroscedastic Bayesian Optimisation

Bayesian optimisation is an important decision-making tool for high-stak...
research
03/09/2015

Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs

Standard sparse pseudo-input approximations to the Gaussian process (GP)...
research
01/12/2020

Bayesian Quantile and Expectile Optimisation

Bayesian optimisation is widely used to optimise stochastic black box fu...
research
10/02/2019

Optimising Optimisers with Push GP

This work uses Push GP to automatically design both local and population...
research
09/08/2020

Sequential Subspace Search for Functional Bayesian Optimization Incorporating Experimenter Intuition

We propose an algorithm for Bayesian functional optimisation - that is, ...

Please sign up or login with your details

Forgot password? Click here to reset