Learning the Structure of Deep Sparse Graphical Models

12/31/2009
by   Ryan Prescott Adams, et al.
0

Deep belief networks are a powerful way to model complex probability distributions. However, learning the structure of a belief network, particularly one with hidden units, is difficult. The Indian buffet process has been used as a nonparametric Bayesian prior on the directed structure of a belief network with a single infinitely wide hidden layer. In this paper, we introduce the cascading Indian buffet process (CIBP), which provides a nonparametric prior on the structure of a layered, directed belief network that is unbounded in both depth and width, yet allows tractable inference. We use the CIBP prior with the nonlinear Gaussian belief network so each unit can additionally vary its behavior between discrete and continuous representations. We provide Markov chain Monte Carlo algorithms for inference in these belief networks and explore the structures learned on several image data sets.

READ FULL TEXT

page 13

page 14

page 15

research
06/20/2023

A Bayesian Take on Gaussian Process Networks

Gaussian Process Networks (GPNs) are a class of directed graphical model...
research
11/05/2012

Kernels and Submodels of Deep Belief Networks

We study the mixtures of factorizing probability distributions represent...
research
10/04/2013

Labeled Directed Acyclic Graphs: a generalization of context-specific independence in directed graphical models

We introduce a novel class of labeled directed acyclic graph (LDAG) mode...
research
12/09/2015

Gamma Belief Networks

To infer multilayer deep representations of high-dimensional discrete an...
research
08/22/2013

Learning Deep Representation Without Parameter Inference for Nonlinear Dimensionality Reduction

Unsupervised deep learning is one of the most powerful representation le...
research
03/06/2013

Minimal Assumption Distribution Propagation in Belief Networks

As belief networks are used to model increasingly complex situations, th...
research
03/27/2013

Ideal Reformulation of Belief Networks

The intelligent reformulation or restructuring of a belief network can g...

Please sign up or login with your details

Forgot password? Click here to reset