A Deep and Tractable Density Estimator

10/07/2013
by   Benigno Uria, et al.
0

The Neural Autoregressive Distribution Estimator (NADE) and its real-valued version RNADE are competitive density models of multidimensional data across a variety of domains. These models use a fixed, arbitrary ordering of the data dimensions. One can easily condition on variables at the beginning of the ordering, and marginalize out variables at the end of the ordering, however other inference tasks require approximate inference. In this work we introduce an efficient procedure to simultaneously train a NADE model for each possible ordering of the variables, by sharing parameters across all these models. We can thus use the most convenient model for each inference task at hand, and ensembles of such models with different orderings are immediately available. Moreover, unlike the original NADE, our training procedure scales to deep models. Empirically, ensembles of Deep NADE models obtain state of the art density estimation performance.

READ FULL TEXT

page 6

page 7

page 8

research
05/07/2016

Neural Autoregressive Distribution Estimation

We present Neural Autoregressive Distribution Estimation (NADE) models, ...
research
06/02/2013

RNADE: The real-valued neural autoregressive density-estimator

We introduce RNADE, a new model for joint density estimation of real-val...
research
04/20/2020

Roundtrip: A Deep Generative Neural Density Estimator

Density estimation is a fundamental problem in both statistics and machi...
research
10/17/2021

PixelPyramids: Exact Inference Models from Lossless Image Pyramids

Autoregressive models are a class of exact inference approaches with hig...
research
06/08/2021

Marginalizable Density Models

Probability density models based on deep networks have achieved remarkab...
research
07/07/2020

The ordering of future observations from multiple groups

There are many situations where comparison of different groups is of gre...
research
04/09/2013

Symmetry-Aware Marginal Density Estimation

The Rao-Blackwell theorem is utilized to analyze and improve the scalabi...

Please sign up or login with your details

Forgot password? Click here to reset