A Deep and Tractable Density Estimator

10/07/2013
by   Benigno Uria, et al.
0

The Neural Autoregressive Distribution Estimator (NADE) and its real-valued version RNADE are competitive density models of multidimensional data across a variety of domains. These models use a fixed, arbitrary ordering of the data dimensions. One can easily condition on variables at the beginning of the ordering, and marginalize out variables at the end of the ordering, however other inference tasks require approximate inference. In this work we introduce an efficient procedure to simultaneously train a NADE model for each possible ordering of the variables, by sharing parameters across all these models. We can thus use the most convenient model for each inference task at hand, and ensembles of such models with different orderings are immediately available. Moreover, unlike the original NADE, our training procedure scales to deep models. Empirically, ensembles of Deep NADE models obtain state of the art density estimation performance.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

page 7

page 8

05/07/2016

Neural Autoregressive Distribution Estimation

We present Neural Autoregressive Distribution Estimation (NADE) models, ...
06/02/2013

RNADE: The real-valued neural autoregressive density-estimator

We introduce RNADE, a new model for joint density estimation of real-val...
04/20/2020

Roundtrip: A Deep Generative Neural Density Estimator

Density estimation is a fundamental problem in both statistics and machi...
06/08/2021

Marginalizable Density Models

Probability density models based on deep networks have achieved remarkab...
07/07/2020

The ordering of future observations from multiple groups

There are many situations where comparison of different groups is of gre...
11/15/2015

Mixtures of Sparse Autoregressive Networks

We consider high-dimensional distribution estimation through autoregress...
10/31/2017

Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering

Existing deep multitask learning (MTL) approaches align layers shared be...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.