Calibration Improves Bayesian Optimization

12/08/2021
by   Shachi Deshpande, et al.
0

Bayesian optimization is a procedure that allows obtaining the global optimum of black-box functions and that is useful in applications such as hyper-parameter optimization. Uncertainty estimates over the shape of the objective function are instrumental in guiding the optimization process. However, these estimates can be inaccurate if the objective function violates assumptions made within the underlying model (e.g., Gaussianity). We propose a simple algorithm to calibrate the uncertainty of posterior distributions over the objective function as part of the Bayesian optimization process. We show that by improving the uncertainty estimates of the posterior distribution with calibration, Bayesian optimization makes better decisions and arrives at the global optimum in fewer steps. We show that this technique improves the performance of Bayesian optimization on standard benchmark functions and hyperparameter optimization tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/06/2023

Random postprocessing for combinatorial Bayesian optimization

Model-based sequential approaches to discrete "black-box" optimization, ...
research
10/22/2022

Bayesian Optimization with Conformal Coverage Guarantees

Bayesian optimization is a coherent, ubiquitous approach to decision-mak...
research
02/26/2020

Incorporating Expert Prior Knowledge into Experimental Design via Posterior Sampling

Scientific experiments are usually expensive due to complex experimental...
research
09/22/2021

A Robust Asymmetric Kernel Function for Bayesian Optimization, with Application to Image Defect Detection in Manufacturing Systems

Some response surface functions in complex engineering systems are usual...
research
01/14/2023

On the role of Model Uncertainties in Bayesian Optimization

Bayesian optimization (BO) is a popular method for black-box optimizatio...
research
02/26/2021

NOMU: Neural Optimization-based Model Uncertainty

We introduce a new approach for capturing model uncertainty for neural n...
research
07/13/2022

Non-Myopic Multifidelity Bayesian Optimization

Bayesian optimization is a popular framework for the optimization of bla...

Please sign up or login with your details

Forgot password? Click here to reset