Diversified Sampling for Batched Bayesian Optimization with Determinantal Point Processes

10/22/2021
by   Elvis Nava, et al.
0

In Bayesian Optimization (BO) we study black-box function optimization with noisy point evaluations and Bayesian priors. Convergence of BO can be greatly sped up by batching, where multiple evaluations of the black-box function are performed in a single round. The main difficulty in this setting is to propose at the same time diverse and informative batches of evaluation points. In this work, we introduce DPP-Batch Bayesian Optimization (DPP-BBO), a universal framework for inducing batch diversity in sampling based BO by leveraging the repulsive properties of Determinantal Point Processes (DPP) to naturally diversify the batch sampling procedure. We illustrate this framework by formulating DPP-Thompson Sampling (DPP-TS) as a variant of the popular Thompson Sampling (TS) algorithm and introducing a Markov Chain Monte Carlo procedure to sample from it. We then prove novel Bayesian simple regret bounds for both classical batched TS as well as our counterpart DPP-TS, with the latter bound being tighter. Our real-world, as well as synthetic, experiments demonstrate improved performance of DPP-BBO over classical batching methods with Gaussian process and Cox process models.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 23

11/04/2019

On Batch Bayesian Optimization

We present two algorithms for Bayesian optimization in the batch feedbac...
05/24/2019

Efficient Batch Black-box Optimization with Deterministic Regret Bounds

In this work, we investigate black-box optimization from the perspective...
03/10/2020

Time-varying Gaussian Process Bandit Optimization with Non-constant Evaluation Time

The Gaussian process bandit is a problem in which we want to find a maxi...
04/21/2020

Bayesian Optimization of Hyperparameters when the Marginal Likelihood is Estimated by MCMC

Bayesian models often involve a small set of hyperparameters determined ...
06/08/2021

Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization

Bayesian optimization (BO) conventionally relies on handcrafted acquisit...
06/21/2017

Constrained Bayesian Optimization with Noisy Experiments

Randomized experiments are the gold standard for evaluating the effects ...
05/31/2019

Adaptive Model Refinement with Batch Bayesian Sampling for Optimization of Bio-inspired Flow Tailoring

This paper presents an advancement to an approach for model-independent ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.