Dependent Multinomial Models Made Easy: Stick Breaking with the Pólya-Gamma Augmentation

06/18/2015
by   Scott W. Linderman, et al.
0

Many practical modeling problems involve discrete data that are best represented as draws from multinomial or categorical distributions. For example, nucleotides in a DNA sequence, children's names in a given state and year, and text documents are all commonly modeled with multinomial distributions. In all of these cases, we expect some form of dependency between the draws: the nucleotide at one position in the DNA strand may depend on the preceding nucleotides, children's names are highly correlated from year to year, and topics in text may be correlated and dynamic. These dependencies are not naturally captured by the typical Dirichlet-multinomial formulation. Here, we leverage a logistic stick-breaking representation and recent innovations in Pólya-gamma augmentation to reformulate the multinomial distribution in terms of latent variables with jointly Gaussian likelihoods, enabling us to take advantage of a host of Bayesian inference techniques for Gaussian models with minimal overhead.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/29/2019

Bayesian Inference for Polya Inverse Gamma Models

Probability density functions that include the gamma function are widely...
research
05/02/2012

Bayesian inference for logistic models using Polya-Gamma latent variables

We propose a new data-augmentation strategy for fully Bayesian inference...
research
06/03/2021

Bayesian Inference for Gamma Models

We use the theory of normal variance-mean mixtures to derive a data augm...
research
03/17/2022

Efficient dependency models for some distributions

Dependency functions of dependent variables are relevant for i) performi...
research
07/17/2018

Multimatricvariate distribution under elliptical models

A new family of matrix variate distributions indexed by elliptical model...
research
11/13/2020

Ultimate Pólya Gamma Samplers – Efficient MCMC for possibly imbalanced binary and categorical data

Modeling binary and categorical data is one of the most commonly encount...
research
04/16/2016

Smoothed Hierarchical Dirichlet Process: A Non-Parametric Approach to Constraint Measures

Time-varying mixture densities occur in many scenarios, for example, the...

Please sign up or login with your details

Forgot password? Click here to reset