Hybrid Batch Bayesian Optimization

by   Javad Azimi, et al.

Bayesian Optimization aims at optimizing an unknown non-convex/concave function that is costly to evaluate. We are interested in application scenarios where concurrent function evaluations are possible. Under such a setting, BO could choose to either sequentially evaluate the function, one input at a time and wait for the output of the function before making the next selection, or evaluate the function at a batch of multiple inputs at once. These two different settings are commonly referred to as the sequential and batch settings of Bayesian Optimization. In general, the sequential setting leads to better optimization performance as each function evaluation is selected with more information, whereas the batch setting has an advantage in terms of the total experimental time (the number of iterations). In this work, our goal is to combine the strength of both settings. Specifically, we systematically analyze Bayesian optimization using Gaussian process as the posterior estimator and provide a hybrid algorithm that, based on the current state, dynamically switches between a sequential policy and a batch policy with variable batch sizes. We provide theoretical justification for our algorithm and present experimental results on eight benchmark BO problems. The results show that our method achieves substantial speedup (up to policy, without suffering any significant performance loss.


page 1

page 2

page 3

page 4


On Batch Bayesian Optimization

We present two algorithms for Bayesian optimization in the batch feedbac...

Combining Multi-Fidelity Modelling and Asynchronous Batch Bayesian Optimization

Bayesian Optimization is a useful tool for experiment design. Unfortunat...

Exploiting gradients and Hessians in Bayesian optimization and Bayesian quadrature

An exciting branch of machine learning research focuses on methods for l...

Efficient nonmyopic Bayesian optimization and quadrature

Finite-horizon sequential decision problems arise naturally in many mach...

Sequential Decision Making on Unmatched Data using Bayesian Kernel Embeddings

The problem of sequentially maximizing the expectation of a function see...

Bayesian Optimization With Censored Response Data

Bayesian optimization (BO) aims to minimize a given blackbox function us...

Batched Stochastic Bayesian Optimization via Combinatorial Constraints Design

In many high-throughput experimental design settings, such as those comm...

Please sign up or login with your details

Forgot password? Click here to reset