Measuring Sample Quality with Kernels

by   Jackson Gorham, et al.

Approximate Markov chain Monte Carlo (MCMC) offers the promise of more rapid sampling at the cost of more biased inference. Since standard MCMC diagnostics fail to detect these biases, researchers have developed computable Stein discrepancy measures that provably determine the convergence of a sample to its target distribution. This approach was recently combined with the theory of reproducing kernels to define a closed-form kernel Stein discrepancy (KSD) computable by summing kernel evaluations across pairs of sample points. We develop a theory of weak convergence for KSDs based on Stein's method, demonstrate that commonly used KSDs fail to detect non-convergence even for Gaussian targets, and show that kernels with slowly decaying tails provably determine convergence for a large class of target distributions. The resulting convergence-determining KSDs are suitable for comparing biased, exact, and deterministic sample sequences and simpler to compute and parallelize than alternative Stein discrepancies. We use our tools to compare biased samplers, select sampler hyperparameters, and improve upon existing KSD approaches to one-sample hypothesis testing and sample quality improvement.



There are no comments yet.


page 1

page 2

page 3

page 4


Measuring Sample Quality with Diffusions

Standard Markov chain Monte Carlo diagnostics, like effective sample siz...

Kernelized Complete Conditional Stein Discrepancy

Much of machine learning relies on comparing distributions with discrepa...

Random Feature Stein Discrepancies

Computable Stein discrepancies have been deployed for a variety of appli...

Measuring Sample Quality with Stein's Method

To improve the efficiency of Monte Carlo estimation, practitioners are t...

The reproducing Stein kernel approach for post-hoc corrected sampling

Stein importance sampling is a widely applicable technique based on kern...

Nonreversible MCMC from conditional invertible transforms: a complete recipe with convergence guarantees

Markov Chain Monte Carlo (MCMC) is a class of algorithms to sample compl...

Zonotope hit-and-run for efficient sampling from projection DPPs

Determinantal point processes (DPPs) are distributions over sets of item...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.