Autoregressive Energy Machines

by   Charlie Nash, et al.

Neural density estimators are flexible families of parametric models which have seen widespread use in unsupervised machine learning in recent years. Maximum-likelihood training typically dictates that these models be constrained to specify an explicit density. However, this limitation can be overcome by instead using a neural network to specify an energy function, or unnormalized density, which can subsequently be normalized to obtain a valid distribution. The challenge with this approach lies in accurately estimating the normalizing constant of the high-dimensional energy function. We propose the Autoregressive Energy Machine, an energy-based model which simultaneously learns an unnormalized density and computes an importance-sampling estimate of the normalizing constant for each conditional in an autoregressive decomposition. The Autoregressive Energy Machine achieves state-of-the-art performance on a suite of density-estimation tasks.


page 1

page 5

page 8


Masked Autoregressive Flow for Density Estimation

Autoregressive models are among the best performing neural density estim...

ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation

We propose to train a non-autoregressive machine translation model to mi...

Transformation Autoregressive Networks

The fundamental task of general density estimation has been of keen inte...

Autoregressive Models: What Are They Good For?

Autoregressive (AR) models have become a popular tool for unsupervised l...

Arbitrary Conditional Distributions with Energy

Modeling distributions of covariates, or density estimation, is a core c...

Variable Skipping for Autoregressive Range Density Estimation

Deep autoregressive models compute point likelihood estimates of individ...

Learning Discrete Distributions by Dequantization

Media is generally stored digitally and is therefore discrete. Many succ...

Code Repositories


Autoregressive Energy Machines

view repo

Please sign up or login with your details

Forgot password? Click here to reset