DeepAI
Log In Sign Up

Deep Temporal Sigmoid Belief Networks for Sequence Modeling

09/23/2015
by   Zhe Gan, et al.
0

Deep dynamic generative models are developed to learn sequential dependencies in time-series data. The multi-layered model is designed by constructing a hierarchy of temporal sigmoid belief networks (TSBNs), defined as a sequential stack of sigmoid belief networks (SBNs). Each SBN has a contextual hidden state, inherited from the previous SBNs in the sequence, and is used to regulate its hidden bias. Scalable learning and inference algorithms are derived by introducing a recognition model that yields fast sampling from the variational posterior. This recognition model is trained jointly with the generative model, by maximizing its variational lower bound on the log-likelihood. Experimental results on bouncing balls, polyphonic music, motion capture, and text streams show that the proposed approach achieves state-of-the-art predictive performance, and has the capacity to synthesize various sequences.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/22/2016

Factored Temporal Sigmoid Belief Networks for Sequence Learning

Deep conditional generative models are developed to simultaneously learn...
01/31/2014

Neural Variational Inference and Learning in Belief Networks

Highly expressive directed latent variable models, such as sigmoid belie...
02/06/2016

Ladder Variational Autoencoders

Variational Autoencoders are powerful models for unsupervised learning. ...
09/28/2020

Variational Temporal Deep Generative Model for Radar HRRP Target Recognition

We develop a recurrent gamma belief network (rGBN) for radar automatic t...
12/21/2017

Truncated Variational Sampling for "Black Box" Optimization of Generative Models

We investigate the optimization of two generative models with binary hid...
09/12/2013

Temporal Autoencoding Improves Generative Models of Time Series

Restricted Boltzmann Machines (RBMs) are generative models which can lea...

Code Repositories

TSBN_code_NIPS2015

The Matlab Code and the Supplementary Material for the NIPS 2015 paper "Deep Temporal Sigmoid Belief Networks for Sequence Modeling"


view repo

SBN_NVIL

Neural Variational Inference and Learning for Sigmoid Belief Network


view repo