Blocking Collapsed Gibbs Sampler for Latent Dirichlet Allocation Models

08/02/2016
by   Xin Zhang, et al.
0

The latent Dirichlet allocation (LDA) model is a widely-used latent variable model in machine learning for text analysis. Inference for this model typically involves a single-site collapsed Gibbs sampling step for latent variables associated with observations. The efficiency of the sampling is critical to the success of the model in practical large scale applications. In this article, we introduce a blocking scheme to the collapsed Gibbs sampler for the LDA model which can, with a theoretical guarantee, improve chain mixing efficiency. We develop two procedures, an O(K)-step backward simulation and an O(log K)-step nested simulation, to directly sample the latent variables within each block. We demonstrate that the blocking scheme achieves substantial improvements in chain mixing compared to the state of the art single-site collapsed Gibbs sampler. We also show that when the number of topics is over hundreds, the nested-simulation blocking scheme can achieve a significant reduction in computation time compared to the single-site sampler.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/27/2018

Adaptive Scan Gibbs Sampler for Large Scale Inference Problems

For large scale on-line inference problems the update strategy is critic...
research
09/22/2020

The computational cost of blocking for sampling discretely observed diffusions

Many approaches for conducting Bayesian inference on discretely observed...
research
08/04/2014

Modulation Classification via Gibbs Sampling Based on a Latent Dirichlet Bayesian Network

A novel Bayesian modulation classification scheme is proposed for a sing...
research
06/11/2015

Sparse Partially Collapsed MCMC for Parallel Inference in Topic Models

Topic models, and more specifically the class of Latent Dirichlet Alloca...
research
04/11/2018

Interdependent Gibbs Samplers

Gibbs sampling, as a model learning method, is known to produce the most...
research
09/11/2021

Microbiome subcommunity learning with logistic-tree normal latent Dirichlet allocation

Mixed-membership (MM) models such as Latent Dirichlet Allocation (LDA) h...
research
07/18/2021

Gibbs sampling for mixtures in order of appearance: the ordered allocation sampler

Gibbs sampling methods for mixture models are based on data augmentation...

Please sign up or login with your details

Forgot password? Click here to reset