Approximate Sampling using an Accelerated Metropolis-Hastings based on Bayesian Optimization and Gaussian Processes

10/21/2019

∙

Markov Chain Monte Carlo (MCMC) methods have a drawback when working with a target distribution or likelihood function that is computationally expensive to evaluate, specially when working with big data. This paper focuses on Metropolis-Hastings (MH) algorithm for unimodal distributions. Here, an enhanced MH algorithm is proposed that requires less number of expensive function evaluations, has shorter burn-in period, and uses a better proposal distribution. The main innovations include the use of Bayesian optimization to reach the high probability region quickly, emulating the target distribution using Gaussian processes (GP), and using Laplace approximation of the GP to build a proposal distribution that captures the underlying correlation better. The experiments show significant improvement over the regular MH. Statistical comparison between the results from two algorithms is presented.

READ FULL TEXT

Approximate Sampling using an Accelerated Metropolis-Hastings based on Bayesian Optimization and Gaussian Processes

Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes

Estimation of Failure Probabilities via Local Subset Approximations

Iterative Construction of Gaussian Process Surrogate Models for Bayesian Inference

Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect

Practical Hilbert space approximate Bayesian Gaussian processes for probabilistic programming

Bayesian Optimization of Hyperparameters when the Marginal Likelihood is Estimated by MCMC

Accelerating ABC methods using Gaussian processes

Approximate Sampling using an Accelerated Metropolis-Hastings based on Bayesian Optimization and Gaussian Processes

Related Research

Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes

Estimation of Failure Probabilities via Local Subset Approximations

Iterative Construction of Gaussian Process Surrogate Models for Bayesian Inference

Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect

Practical Hilbert space approximate Bayesian Gaussian processes for probabilistic programming

Bayesian Optimization of Hyperparameters when the Marginal Likelihood is Estimated by MCMC

Accelerating ABC methods using Gaussian processes