Scaling Gaussian Process Optimization by Evaluating a Few Unique Candidates Multiple Times

01/30/2022
by   Daniele Calandriello, et al.
0

Computing a Gaussian process (GP) posterior has a computational cost cubical in the number of historical points. A reformulation of the same GP posterior highlights that this complexity mainly depends on how many unique historical points are considered. This can have important implication in active learning settings, where the set of historical points is constructed sequentially by the learner. We show that sequential black-box optimization based on GPs (GP-Opt) can be made efficient by sticking to a candidate solution for multiple evaluation steps and switch only when necessary. Limiting the number of switches also limits the number of unique points in the history of the GP. Thus, the efficient GP reformulation can be used to exactly and cheaply compute the posteriors required to run the GP-Opt algorithms. This approach is especially useful in real-world applications of GP-Opt with high switch costs (e.g. switching chemicals in wet labs, data/model loading in hyperparameter optimization). As examples of this meta-approach, we modify two well-established GP-Opt algorithms, GP-UCB and GP-EI, to switch candidates as infrequently as possible adapting rules from batched GP-Opt. These versions preserve all the theoretical no-regret guarantees while improving practical aspects of the algorithms such as runtime, memory complexity, and the ability of batching candidates and evaluating them in parallel.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/23/2020

Near-linear Time Gaussian Process Optimization with Adaptive Batching and Resparsification

Gaussian processes (GP) are one of the most successful frameworks to mod...
research
03/13/2019

Gaussian Process Optimization with Adaptive Sketching: Scalable and No Regret

Gaussian processes (GP) are a popular Bayesian approach for the optimiza...
research
05/22/2022

Fast Gaussian Process Posterior Mean Prediction via Local Cross Validation and Precomputation

Gaussian processes (GPs) are Bayesian non-parametric models useful in a ...
research
06/14/2022

On Provably Robust Meta-Bayesian Optimization

Bayesian optimization (BO) has become popular for sequential optimizatio...
research
06/16/2021

Ada-BKB: Scalable Gaussian Process Optimization on Continuous Domain by Adaptive Discretization

Gaussian process optimization is a successful class of algorithms (e.g. ...
research
10/20/2016

Gaussian process modeling in approximate Bayesian computation to estimate horizontal gene transfer in bacteria

Approximate Bayesian computation (ABC) can be used for model fitting whe...
research
02/28/2023

Interactive Segmentation as Gaussian Process Classification

Click-based interactive segmentation (IS) aims to extract the target obj...

Please sign up or login with your details

Forgot password? Click here to reset