Integrals over Gaussians under Linear Domain Constraints

10/21/2019
by   Alexandra Gessner, et al.
0

Integrals of linearly constrained multivariate Gaussian densities are a frequent problem in machine learning and statistics, arising in tasks like generalized linear models and Bayesian optimization. Yet they are notoriously hard to compute, and to further complicate matters, the numerical values of such integrals may be very small. We present an efficient black-box algorithm that exploits geometry for the estimation of integrals over a small, truncated Gaussian volume, and to simulate therefrom. Our algorithm uses the Holmes-Diaconis-Ross (HDR) method combined with an analytic version of elliptical slice sampling (ESS). Adapted to the linear setting, ESS allows for efficient, rejection-free sampling, because intersections of ellipses and domain boundaries have closed-form solutions. The key idea of HDR is to decompose the integral into easier-to-compute conditional probabilities by using a sequence of nested domains. Remarkably, it allows for direct computation of the logarithm of the integral value and thus enables the computation of extremely small probability masses. We demonstrate the effectiveness of our tailored combination of HDR and ESS on high-dimensional integrals and on entropy search for Bayesian optimization.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/08/2020

Projective Preferential Bayesian Optimization

Bayesian optimization is an effective method for finding extrema of a bl...
03/06/2017

Batched High-dimensional Bayesian Optimization via Structural Kernel Learning

Optimization of high-dimensional black-box functions is an extremely cha...
12/17/2021

Nested Bayesian Optimization for Computer Experiments

Computer experiments can emulate the physical systems, help computationa...
06/04/2019

Bayesian Optimization of Composite Functions

We consider optimization of composite objective functions, i.e., of the ...
09/09/2016

Efficient batch-sequential Bayesian optimization with moments of truncated Gaussian vectors

We deal with the efficient parallelization of Bayesian global optimizati...
11/17/2015

Bayesian Optimization with Dimension Scheduling: Application to Biological Systems

Bayesian Optimization (BO) is a data-efficient method for global black-b...
10/20/2020

Federated Bayesian Optimization via Thompson Sampling

Bayesian optimization (BO) is a prominent approach to optimizing expensi...