Lookahead Bayesian Optimization via Rollout: Guarantees and Sequential Rolling Horizons

by   Xubo Yue, et al.

Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic programming (DP) formulation that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-specified model. In this work we focus on the rollout approximation for solving the intractable DP. We first prove the improving nature of rollout in tackling lookahead BO. We then provide both a theoretical and practical guideline to decide on the rolling horizon stagewise. This guideline is built on quantifying the negative effect of a mis-specified model. To illustrate our idea, we provide case studies on both single and multi-information source BO. Empirical results show the advantageous properties of our method over several myopic and non-myopic BO algorithms.



There are no comments yet.


page 1

page 2

page 3

page 4


Efficient nonmyopic Bayesian optimization and quadrature

Finite-horizon sequential decision problems arise naturally in many mach...

Learning to Stop with Surprisingly Few Samples

We consider a discounted infinite horizon optimal stopping problem. If t...

Deep neural networks algorithms for stochastic control problems on finite horizon, part I: convergence analysis

This paper develops algorithms for high-dimensional stochastic control p...

Robustness and risk management via distributional dynamic programming

In dynamic programming (DP) and reinforcement learning (RL), an agent le...

Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees

Bayesian optimization is a sequential decision making framework for opti...

Epidemiologically and Socio-economically Optimal Policies via Bayesian Optimization

Mass public quarantining, colloquially known as a lock-down, is a non-ph...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.