Prior-guided Bayesian Optimization

06/25/2020
by   Artur Souza, et al.
53

While Bayesian Optimization (BO) is a very popular method for optimizing expensive black-box functions, it fails to leverage the experience of domain experts. This causes BO to waste function evaluations on commonly known bad regions of design choices, e.g., hyperparameters of a machine learning algorithm. To address this issue, we introduce Prior-guided Bayesian Optimization (PrBO). PrBO allows users to inject their knowledge into the optimization process in the form of priors about which parts of the input space will yield the best performance, rather than BO's standard priors over functions which are much less intuitive for users. PrBO then combines these priors with BO's standard probabilistic model to yield a posterior. We show that PrBO is more sample efficient than state-of-the-art methods without user priors and 10,000× faster than random search, on a common suite of benchmarks and a real-world hardware design application. We also show that PrBO converges faster even if the user priors are not entirely accurate and that it robustly recovers from misleading priors.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/21/2020

Towards Automatic Bayesian Optimization: A first step involving acquisition functions

Bayesian Optimization is the state of the art technique for the optimiza...
09/20/2017

Bayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Search

One of the most interesting features of Bayesian optimization for direct...
12/11/2018

Doubly Bayesian Optimization

Bayesian optimization (BO) is a powerful method for optimizing complex b...
03/02/2015

Bayesian Optimization of Text Representations

When applying machine learning to problems in NLP, there are many choice...
07/07/2022

Pre-training helps Bayesian optimization too

Bayesian optimization (BO) has become a popular strategy for global opti...
04/23/2022

πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization

Bayesian optimization (BO) has become an established framework and popul...
12/02/2021

Invariant Priors for Bayesian Quadrature

Bayesian quadrature (BQ) is a model-based numerical integration method t...