Bayesian Analysis of Dynamic Linear Topic Models

by   Chris Glynn, et al.

In dynamic topic modeling, the proportional contribution of a topic to a document depends on the temporal dynamics of that topic's overall prevalence in the corpus. We extend the Dynamic Topic Model of Blei and Lafferty (2006) by explicitly modeling document level topic proportions with covariates and dynamic structure that includes polynomial trends and periodicity. A Markov Chain Monte Carlo (MCMC) algorithm that utilizes Polya-Gamma data augmentation is developed for posterior inference. Conditional independencies in the model and sampling are made explicit, and our MCMC algorithm is parallelized where possible to allow for inference in large corpora. To address computational bottlenecks associated with Polya-Gamma sampling, we appeal to the Central Limit Theorem to develop a Gaussian approximation to the Polya-Gamma random variable. This approximation is fast and reliable for parameter values relevant in the text mining domain. Our model and inference algorithm are validated with multiple simulation examples, and we consider the application of modeling trends in PubMed abstracts. We demonstrate that sharing information across documents is critical for accurately estimating document-specific topic proportions. We also show that explicitly modeling polynomial and periodic behavior improves our ability to predict topic prevalence at future time points.


page 8

page 16

page 19

page 26


Bayesian Non-Homogeneous Markov Models via Polya-Gamma Data Augmentation with Applications to Rainfall Modeling

Discrete-time hidden Markov models are a broadly useful class of latent-...

A Distributed Algorithm for Polya-Gamma Data Augmentation

The Polya-Gamma data augmentation (PG-DA) algorithm is routinely used fo...

Analyses of Multi-collection Corpora via Compound Topic Modeling

As electronically stored data grow in daily life, obtaining novel and re...

A Spatiotemporal Gamma Shot Noise Cox Process

A new discrete-time shot noise Cox process for spatiotemporal data is pr...

Infinite Author Topic Model based on Mixed Gamma-Negative Binomial Process

Incorporating the side information of text corpus, i.e., authors, time s...

Locally Adaptive Dynamic Networks

Our focus is on realistically modeling and forecasting dynamic networks ...

Bayesian Sparse Blind Deconvolution Using MCMC Methods Based on Normal-Inverse-Gamma Prior

Bayesian estimation methods for sparse blind deconvolution problems conv...

Please sign up or login with your details

Forgot password? Click here to reset