Global Optimization of Gaussian processes

05/21/2020
by   Artur M. Schweidtmann, et al.
0

Gaussian processes (Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the degrees of freedom and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the "MeLOn - Machine Learning Models for Optimization" toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

READ FULL TEXT
research
05/07/2019

Bayesian Optimization using Deep Gaussian Processes

Bayesian Optimization using Gaussian Processes is a popular approach to ...
research
11/16/2021

Accounting for Gaussian Process Imprecision in Bayesian Optimization

Bayesian optimization (BO) with Gaussian processes (GP) as surrogate mod...
research
06/25/2020

Green Machine Learning via Augmented Gaussian Processes and Multi-Information Source Optimization

Searching for accurate Machine and Deep Learning models is a computation...
research
11/09/2021

Optimizing Bayesian acquisition functions in Gaussian Processes

Bayesian Optimization is an effective method for searching the global ma...
research
08/16/2016

Fast Calculation of the Knowledge Gradient for Optimization of Deterministic Engineering Simulations

A novel efficient method for computing the Knowledge-Gradient policy for...
research
08/04/2022

Bayesian Optimization with Informative Covariance

Bayesian Optimization is a methodology for global optimization of unknow...
research
05/22/2018

Neural Generative Models for Global Optimization with Gradients

The aim of global optimization is to find the global optimum of arbitrar...

Please sign up or login with your details

Forgot password? Click here to reset