Lookahead Bayesian Optimization via Rollout: Guarantees and Sequential Rolling Horizons

11/04/2019
by   Xubo Yue, et al.
0

Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to find optimal sampling policies through solving a dynamic programming (DP) formulation that maximizes a long-term reward over a rolling horizon. Though promising, lookahead BO faces the risk of error propagation through its increased dependence on a possibly mis-specified model. In this work we focus on the rollout approximation for solving the intractable DP. We first prove the improving nature of rollout in tackling lookahead BO. We then provide both a theoretical and practical guideline to decide on the rolling horizon stagewise. This guideline is built on quantifying the negative effect of a mis-specified model. To illustrate our idea, we provide case studies on both single and multi-information source BO. Empirical results show the advantageous properties of our method over several myopic and non-myopic BO algorithms.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

09/10/2019

Efficient nonmyopic Bayesian optimization and quadrature

Finite-horizon sequential decision problems arise naturally in many mach...
02/19/2021

Learning to Stop with Surprisingly Few Samples

We consider a discounted infinite horizon optimal stopping problem. If t...
12/11/2018

Deep neural networks algorithms for stochastic control problems on finite horizon, part I: convergence analysis

This paper develops algorithms for high-dimensional stochastic control p...
12/28/2021

Robustness and risk management via distributional dynamic programming

In dynamic programming (DP) and reinforcement learning (RL), an agent le...
06/29/2020

Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees

Bayesian optimization is a sequential decision making framework for opti...
05/22/2020

Epidemiologically and Socio-economically Optimal Policies via Bayesian Optimization

Mass public quarantining, colloquially known as a lock-down, is a non-ph...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.