Warped Gradient-Enhanced Gaussian Process Surrogate Models for Inference with Intractable Likelihoods

by   Quan Vu, et al.

Markov chain Monte Carlo methods for intractable likelihoods, such as the exchange algorithm, require simulations of the sufficient statistics at every iteration of the Markov chain, which often result in expensive computations. Surrogate models for the likelihood function have been developed to accelerate inference algorithms in this context. However, these surrogate models tend to be relatively inflexible, and often provide a poor approximation to the true likelihood function. In this article, we propose the use of a warped, gradient-enhanced, Gaussian process surrogate model for the likelihood function, which jointly models the sample means and variances of the sufficient statistics, and uses warping functions to capture covariance nonstationarity in the input parameter space. We show that both the consideration of nonstationarity and the inclusion of gradient information can be leveraged to obtain a surrogate model that outperforms the conventional stationary Gaussian process surrogate model when making inference, particularly in regions where the likelihood function exhibits a phase transition. We also show that the proposed surrogate model can be used to improve the effective sample size per unit time when embedded in exact inferential algorithms. The utility of our approach in speeding up inferential algorithms is demonstrated on simulated and real-world data.


Asymptotics of maximum likelihood estimators based on Markov chain Monte Carlo methods

In many complex statistical models maximum likelihood estimators cannot ...

Soft Constraints for Inference with Declarative Knowledge

We develop a likelihood free inference procedure for conditioning a prob...

Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Bases

For big data analysis, high computational cost for Bayesian methods ofte...

Statistical inference of random graphs with a surrogate likelihood function

Spectral estimators have been broadly applied to statistical network ana...

Tempering by Subsampling

In this paper we demonstrate that tempering Markov chain Monte Carlo sam...

Sequential sampling of Gaussian process latent variable models

We consider the problem of inferring a latent function in a probabilisti...

Bayesian Indirect Inference for Models with Intractable Normalizing Functions

Inference for doubly intractable distributions is challenging because th...

Please sign up or login with your details

Forgot password? Click here to reset