Scaling Gaussian Process Optimization by Evaluating a Few Unique Candidates Multiple Times

by   Daniele Calandriello, et al.
Università di Genova

Computing a Gaussian process (GP) posterior has a computational cost cubical in the number of historical points. A reformulation of the same GP posterior highlights that this complexity mainly depends on how many unique historical points are considered. This can have important implication in active learning settings, where the set of historical points is constructed sequentially by the learner. We show that sequential black-box optimization based on GPs (GP-Opt) can be made efficient by sticking to a candidate solution for multiple evaluation steps and switch only when necessary. Limiting the number of switches also limits the number of unique points in the history of the GP. Thus, the efficient GP reformulation can be used to exactly and cheaply compute the posteriors required to run the GP-Opt algorithms. This approach is especially useful in real-world applications of GP-Opt with high switch costs (e.g. switching chemicals in wet labs, data/model loading in hyperparameter optimization). As examples of this meta-approach, we modify two well-established GP-Opt algorithms, GP-UCB and GP-EI, to switch candidates as infrequently as possible adapting rules from batched GP-Opt. These versions preserve all the theoretical no-regret guarantees while improving practical aspects of the algorithms such as runtime, memory complexity, and the ability of batching candidates and evaluating them in parallel.


page 1

page 2

page 3

page 4


Near-linear Time Gaussian Process Optimization with Adaptive Batching and Resparsification

Gaussian processes (GP) are one of the most successful frameworks to mod...

Gaussian Process Optimization with Adaptive Sketching: Scalable and No Regret

Gaussian processes (GP) are a popular Bayesian approach for the optimiza...

Fast Gaussian Process Posterior Mean Prediction via Local Cross Validation and Precomputation

Gaussian processes (GPs) are Bayesian non-parametric models useful in a ...

On Provably Robust Meta-Bayesian Optimization

Bayesian optimization (BO) has become popular for sequential optimizatio...

Ada-BKB: Scalable Gaussian Process Optimization on Continuous Domain by Adaptive Discretization

Gaussian process optimization is a successful class of algorithms (e.g. ...

Interactive Segmentation as Gaussian Process Classification

Click-based interactive segmentation (IS) aims to extract the target obj...

Please sign up or login with your details

Forgot password? Click here to reset