Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes

03/02/2022
by   Felix Jimenez, et al.
0

Bayesian optimization is a technique for optimizing black-box target functions. At the core of Bayesian optimization is a surrogate model that predicts the output of the target function at previously unseen inputs to facilitate the selection of promising input values. Gaussian processes (GPs) are commonly used as surrogate models but are known to scale poorly with the number of observations. We adapt the Vecchia approximation, a popular GP approximation from spatial statistics, to enable scalable high-dimensional Bayesian optimization. We develop several improvements and extensions, including training warped GPs using mini-batch gradient descent, approximate neighbor search, and selecting multiple input values in parallel. We focus on the use of our warped Vecchia GP in trust-region Bayesian optimization via Thompson sampling. On several test functions and on two reinforcement-learning problems, our methods compared favorably to the state of the art.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/23/2021

Scalable and Flexible Deep Bayesian Optimization with Auxiliary Information for Scientific Problems

Bayesian optimization (BO) is a popular paradigm for global optimization...
research
06/18/2020

Likelihood-Free Inference with Deep Gaussian Processes

In recent years, surrogate models have been successfully used in likelih...
research
11/16/2021

Accounting for Gaussian Process Imprecision in Bayesian Optimization

Bayesian optimization (BO) with Gaussian processes (GP) as surrogate mod...
research
01/31/2019

ProBO: a Framework for Using Probabilistic Programming in Bayesian Optimization

Optimizing an expensive-to-query function is a common task in science an...
research
12/08/2017

Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Start

Bayesian optimization (BO) is a model-based approach for gradient-free b...
research
05/31/2023

A Study of Bayesian Neural Network Surrogates for Bayesian Optimization

Bayesian optimization is a highly efficient approach to optimizing objec...
research
02/19/2015

Scalable Bayesian Optimization Using Deep Neural Networks

Bayesian optimization is an effective methodology for the global optimiz...

Please sign up or login with your details

Forgot password? Click here to reset