Multimodal Transitions for Generative Stochastic Networks

12/19/2013
by   Sherjil Ozair, et al.
0

Generative Stochastic Networks (GSNs) have been recently introduced as an alternative to traditional probabilistic modeling: instead of parametrizing the data distribution directly, one parametrizes a transition operator for a Markov chain whose stationary distribution is an estimator of the data generating distribution. The result of training is therefore a machine that generates samples through this Markov chain. However, the previously introduced GSN consistency theorems suggest that in order to capture a wide class of distributions, the transition operator in general should be multimodal, something that has not been done before this paper. We introduce for the first time multimodal transition distributions for GSNs, in particular using models in the NADE family (Neural Autoregressive Density Estimator) as output distributions of the transition operator. A NADE model is related to an RBM (and can thus model multimodal distributions) but its likelihood (and likelihood gradient) can be computed easily. The parameters of the NADE are obtained as a learned function of the previous state of the learned Markov chain. Experiments clearly illustrate the advantage of such multimodal transition distributions over unimodal GSNs.

READ FULL TEXT

page 5

page 7

research
03/18/2015

GSNs : Generative Stochastic Networks

We introduce a novel training principle for probabilistic models that is...
research
11/07/2017

Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net

We propose a novel method to directly learn a stochastic transition oper...
research
03/09/2023

Learning Stationary Markov Processes with Contrastive Adjustment

We introduce a new optimization algorithm, termed contrastive adjustment...
research
12/22/2014

Generative Class-conditional Autoencoders

Recent work by Bengio et al. (2013) proposes a sampling procedure for de...
research
08/24/2018

Spectral thresholding for the estimation of Markov chain transition operators

We consider estimation of the transition operator P of a Markov chain an...
research
12/04/2020

Decentralized State-Dependent Markov Chain Synthesis for Swarm Guidance

This paper introduces a decentralized state-dependent Markov chain synth...
research
10/07/2022

Estimation of the Order of Non-Parametric Hidden Markov Models using the Singular Values of an Integral Operator

We are interested in assessing the order of a finite-state Hidden Markov...

Please sign up or login with your details

Forgot password? Click here to reset