Integrals over Gaussians under Linear Domain Constraints

10/21/2019
by   Alexandra Gessner, et al.
0

Integrals of linearly constrained multivariate Gaussian densities are a frequent problem in machine learning and statistics, arising in tasks like generalized linear models and Bayesian optimization. Yet they are notoriously hard to compute, and to further complicate matters, the numerical values of such integrals may be very small. We present an efficient black-box algorithm that exploits geometry for the estimation of integrals over a small, truncated Gaussian volume, and to simulate therefrom. Our algorithm uses the Holmes-Diaconis-Ross (HDR) method combined with an analytic version of elliptical slice sampling (ESS). Adapted to the linear setting, ESS allows for efficient, rejection-free sampling, because intersections of ellipses and domain boundaries have closed-form solutions. The key idea of HDR is to decompose the integral into easier-to-compute conditional probabilities by using a sequence of nested domains. Remarkably, it allows for direct computation of the logarithm of the integral value and thus enables the computation of extremely small probability masses. We demonstrate the effectiveness of our tailored combination of HDR and ESS on high-dimensional integrals and on entropy search for Bayesian optimization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/08/2020

Projective Preferential Bayesian Optimization

Bayesian optimization is an effective method for finding extrema of a bl...
research
03/10/2023

Gaussian Max-Value Entropy Search for Multi-Agent Bayesian Optimization

We study the multi-agent Bayesian optimization (BO) problem, where multi...
research
12/17/2021

Nested Bayesian Optimization for Computer Experiments

Computer experiments can emulate the physical systems, help computationa...
research
09/09/2016

Efficient batch-sequential Bayesian optimization with moments of truncated Gaussian vectors

We deal with the efficient parallelization of Bayesian global optimizati...
research
03/03/2023

Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings

We consider the problem of optimizing expensive black-box functions over...
research
11/17/2015

Bayesian Optimization with Dimension Scheduling: Application to Biological Systems

Bayesian Optimization (BO) is a data-efficient method for global black-b...

Please sign up or login with your details

Forgot password? Click here to reset