Efficient MCMC Sampling with Dimension-Free Convergence Rate using ADMM-type Splitting

05/23/2019
by   Maxime Vono, et al.
0

Performing exact Bayesian inference for complex models is intractable. Markov chain Monte Carlo (MCMC) algorithms can provide reliable approximations of the posterior distribution but are computationally expensive for large datasets. A standard approach to mitigate this complexity consists of using subsampling techniques or distributing the data across a cluster. However, these approaches are typically unreliable in high-dimensional scenarios. We focus here on an alternative class of MCMC schemes exploiting a splitting strategy akin to the one used by the celebrated ADMM optimization algorithm. These methods, proposed recently in [43, 51], appear to provide empirically state-of-the-art performance. We generalize here these ideas and propose a detailed theoretical study of one of these algorithms known as the Split Gibbs Sampler. Under regularity conditions, we establish explicit dimension-free convergence rates for this scheme using Ricci curvature and coupling ideas. We demonstrate experimentally the excellent performance of these MCMC schemes on various applications.

READ FULL TEXT

page 8

page 37

research
10/23/2020

No Free Lunch for Approximate MCMC

It is widely known that the performance of Markov chain Monte Carlo (MCM...
research
04/16/2018

Split-and-augmented Gibbs sampler - Application to large-scale inference problems

Recently, a new class of Markov chain Monte Carlo (MCMC) algorithms took...
research
01/28/2019

Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasets

Bayesian inference via standard Markov Chain Monte Carlo (MCMC) methods ...
research
04/21/2023

Plug-and-Play split Gibbs sampler: embedding deep generative priors in Bayesian inference

This paper introduces a stochastic plug-and-play (PnP) sampling algorith...
research
03/10/2021

Dimension free convergence rates for Gibbs samplers for Bayesian linear mixed models

The emergence of big data has led to a growing interest in so-called con...
research
06/11/2021

DG-LMC: A Turn-key and Scalable Synchronous Distributed MCMC Algorithm via Langevin Monte Carlo within Gibbs

Performing reliable Bayesian inference on a big data scale is becoming a...
research
05/24/2019

Convergence Guarantees for Adaptive Bayesian Quadrature Methods

Adaptive Bayesian quadrature (ABQ) is a powerful approach to numerical i...

Please sign up or login with your details

Forgot password? Click here to reset