Asymptotically Optimal Exact Minibatch Metropolis-Hastings

by   Ruqi Zhang, et al.

Metropolis-Hastings (MH) is a commonly-used MCMC algorithm, but it can be intractable on large datasets due to requiring computations over the whole dataset. In this paper, we study minibatch MH methods, which instead use subsamples to enable scaling. We observe that most existing minibatch MH methods are inexact (i.e. they may change the target distribution), and show that this inexactness can cause arbitrarily large errors in inference. We propose a new exact minibatch MH method, , which exposes a tunable trade-off between its batch size and its theoretically guaranteed convergence rate. We prove a lower bound on the batch size that any minibatch MH method must use to retain exactness while guaranteeing fast convergence—the first such bound for minibatch MH—and show is asymptotically optimal in terms of the batch size. Empirically, we show outperforms other exact minibatch MH methods on robust linear regression, truncated Gaussian mixtures, and logistic regression.



There are no comments yet.


page 1

page 2

page 3

page 4


Semistochastic Quadratic Bound Methods

Partition functions arise in a variety of settings, including conditiona...

Fast Variance Reduction Method with Stochastic Batch Size

In this paper we study a family of variance reduction methods with rando...

Convergence rates for Metropolis-Hastings algorithms in the Wasserstein distance

We develop necessary conditions for geometrically fast convergence in th...

Stop Wasting My Gradients: Practical SVRG

We present and analyze several strategies for improving the performance ...

Cost-Sensitive Approach to Batch Size Adaptation for Gradient Descent

In this paper, we propose a novel approach to automatically determine th...

Coresets for Scalable Bayesian Logistic Regression

The use of Bayesian methods in large-scale data settings is attractive b...

Adaptive Cholesky Gaussian Processes

We present a method to fit exact Gaussian process models to large datase...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.