Log In Sign Up

Deep Temporal Sigmoid Belief Networks for Sequence Modeling

by   Zhe Gan, et al.

Deep dynamic generative models are developed to learn sequential dependencies in time-series data. The multi-layered model is designed by constructing a hierarchy of temporal sigmoid belief networks (TSBNs), defined as a sequential stack of sigmoid belief networks (SBNs). Each SBN has a contextual hidden state, inherited from the previous SBNs in the sequence, and is used to regulate its hidden bias. Scalable learning and inference algorithms are derived by introducing a recognition model that yields fast sampling from the variational posterior. This recognition model is trained jointly with the generative model, by maximizing its variational lower bound on the log-likelihood. Experimental results on bouncing balls, polyphonic music, motion capture, and text streams show that the proposed approach achieves state-of-the-art predictive performance, and has the capacity to synthesize various sequences.


page 1

page 2

page 3

page 4


Factored Temporal Sigmoid Belief Networks for Sequence Learning

Deep conditional generative models are developed to simultaneously learn...

Neural Variational Inference and Learning in Belief Networks

Highly expressive directed latent variable models, such as sigmoid belie...

Ladder Variational Autoencoders

Variational Autoencoders are powerful models for unsupervised learning. ...

Variational Temporal Deep Generative Model for Radar HRRP Target Recognition

We develop a recurrent gamma belief network (rGBN) for radar automatic t...

Truncated Variational Sampling for "Black Box" Optimization of Generative Models

We investigate the optimization of two generative models with binary hid...

Temporal Autoencoding Improves Generative Models of Time Series

Restricted Boltzmann Machines (RBMs) are generative models which can lea...

Code Repositories


The Matlab Code and the Supplementary Material for the NIPS 2015 paper "Deep Temporal Sigmoid Belief Networks for Sequence Modeling"

view repo


Neural Variational Inference and Learning for Sigmoid Belief Network

view repo