Gamma Processes, Stick-Breaking, and Variational Inference

10/04/2014
by   Anirban Roychowdhury, et al.
0

While most Bayesian nonparametric models in machine learning have focused on the Dirichlet process, the beta process, or their variants, the gamma process has recently emerged as a useful nonparametric prior in its own right. Current inference schemes for models involving the gamma process are restricted to MCMC-based methods, which limits their scalability. In this paper, we present a variational inference framework for models involving gamma process priors. Our approach is based on a novel stick-breaking constructive definition of the gamma process. We prove correctness of this stick-breaking process by using the characterization of the gamma process as a completely random measure (CRM), and we explicitly derive the rate measure of our construction using Poisson process machinery. We also derive error bounds on the truncation of the infinite process required for variational inference, similar to the truncation analyses for other nonparametric models based on the Dirichlet and beta processes. Our representation is then used to derive a variational inference algorithm for a particular Bayesian nonparametric latent structure formulation known as the infinite Gamma-Poisson model, where the latent variables are drawn from a gamma process prior with Poisson likelihoods. Finally, we present results for our algorithms on nonnegative matrix factorization tasks on document corpora, and show that we compare favorably to both sampling-based techniques and variational approaches based on beta-Bernoulli priors.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/15/2011

Beta-Negative Binomial Process and Poisson Factor Analysis

A beta-negative binomial (BNB) process is proposed, leading to a beta-ga...
research
05/26/2019

A unified construction for series representations and finite approximations of completely random measures

Infinite-activity completely random measures (CRMs) have become importan...
research
06/03/2011

Beta processes, stick-breaking, and power laws

The beta-Bernoulli process provides a Bayesian nonparametric prior for m...
research
03/24/2011

The Discrete Infinite Logistic Normal Distribution

We present the discrete infinite logistic normal distribution (DILN), a ...
research
08/23/2023

On a Divergence-Based Prior Analysis of Stick-Breaking Processes

The nonparametric view of Bayesian inference has transformed statistics ...
research
10/14/2015

Embarrassingly Parallel Variational Inference in Nonconjugate Models

We develop a parallel variational inference (VI) procedure for use in da...
research
07/04/2011

On a Rapid Simulation of the Dirichlet Process

We describe a simple and efficient procedure for approximating the Lévy ...

Please sign up or login with your details

Forgot password? Click here to reset