Fully-Asynchronous Distributed Metropolis Sampler with Optimal Speedup

04/01/2019
by   Weiming Feng, et al.
0

The Metropolis-Hastings algorithm is a fundamental Markov chain Monte Carlo (MCMC) method for sampling and inference. With the advent of Big Data, distributed and parallel variants of MCMC methods are attracting increased attention. In this paper, we give a distributed algorithm that can correctly simulate sequential single-site Metropolis chains without any bias in a fully asynchronous message-passing model. Furthermore, if a natural Lipschitz condition is satisfied by the Metropolis filters, our algorithm can simulate N-step Metropolis chains within O(N/n+ n) rounds of asynchronous communications, where n is the number of variables. For sequential single-site dynamics, whose mixing requires Ω(n n) steps, this achieves an optimal linear speedup. For several well-studied important graphical models, including proper graph coloring, hardcore model, and Ising model, the condition for linear speedup is weaker than the respective uniqueness (mixing) conditions. The novel idea in our algorithm is to resolve updates in advance: the local Metropolis filters can be executed correctly before the full information about neighboring spins is available. This achieves optimal parallelism of Metropolis processes without introducing any bias.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/14/2023

Improved Distributed Algorithms for Random Colorings

Markov Chain Monte Carlo (MCMC) algorithms are a widely-used algorithmic...
research
11/07/2021

Simple Parallel Algorithms for Single-Site Dynamics

The single-site dynamics are a canonical class of Markov chains for samp...
research
04/26/2019

Dynamic MCMC Sampling

The Markov chain Monte Carlo (MCMC) methods are the primary tools for sa...
research
02/19/2018

A Simple Parallel and Distributed Sampling Technique: Local Glauber Dynamics

Sampling constitutes an important tool in a variety of areas: from machi...
research
05/31/2018

Bayesian inference in decomposable graphical models using sequential Monte Carlo methods

InthisstudywepresentasequentialsamplingmethodologyforBayesian inference ...
research
02/23/2022

Many processors, little time: MCMC for partitions via optimal transport couplings

Markov chain Monte Carlo (MCMC) methods are often used in clustering sin...
research
11/26/2018

HOGWILD!-Gibbs can be PanAccurate

Asynchronous Gibbs sampling has been recently shown to be fast-mixing an...

Please sign up or login with your details

Forgot password? Click here to reset