Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature

11/03/2014
by   Tom Gunter, et al.
0

We propose a novel sampling framework for inference in probabilistic models: an active learning approach that converges more quickly (in wall-clock time) than Markov chain Monte Carlo (MCMC) benchmarks. The central challenge in probabilistic inference is numerical integration, to average over ensembles of models or unknown (hyper-)parameters (for example to compute the marginal likelihood or a partition function). MCMC has provided approaches to numerical integration that deliver state-of-the-art inference, but can suffer from sample inefficiency and poor convergence diagnostics. Bayesian quadrature techniques offer a model-based solution to such problems, but their uptake has been hindered by prohibitive computation costs. We introduce a warped model for probabilistic integrands (likelihoods) that are known to be non-negative, permitting a cheap active learning scheme to optimally select sample locations. Our algorithm is demonstrated to offer faster convergence (in seconds) relative to simple Monte Carlo and annealed importance sampling on both synthetic and real-world examples.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/04/2018

Batch Selection for Parallelisation of Bayesian Quadrature

Integration over non-negative integrands is a central problem in machine...
research
05/01/2018

Scalable Importance Tempering and Bayesian Variable Selection

We propose a Monte Carlo algorithm to sample from high-dimensional proba...
research
01/28/2021

A Bayesian Model of Cash Bail Decisions

The use of cash bail as a mechanism for detaining defendants pre-trial i...
research
12/12/2019

Normalizing Constant Estimation with Gaussianized Bridge Sampling

Normalizing constant (also called partition function, Bayesian evidence,...
research
02/07/2020

Assessing the Adversarial Robustness of Monte Carlo and Distillation Methods for Deep Bayesian Neural Network Classification

In this paper, we consider the problem of assessing the adversarial robu...
research
11/03/2022

Fast and robust Bayesian Inference using Gaussian Processes with GPry

We present the GPry algorithm for fast Bayesian inference of general (no...
research
02/22/2022

Parallel MCMC Without Embarrassing Failures

Embarrassingly parallel Markov Chain Monte Carlo (MCMC) exploits paralle...

Please sign up or login with your details

Forgot password? Click here to reset