Autoregressive Energy Machines

04/11/2019
by   Charlie Nash, et al.
42

Neural density estimators are flexible families of parametric models which have seen widespread use in unsupervised machine learning in recent years. Maximum-likelihood training typically dictates that these models be constrained to specify an explicit density. However, this limitation can be overcome by instead using a neural network to specify an energy function, or unnormalized density, which can subsequently be normalized to obtain a valid distribution. The challenge with this approach lies in accurately estimating the normalizing constant of the high-dimensional energy function. We propose the Autoregressive Energy Machine, an energy-based model which simultaneously learns an unnormalized density and computes an importance-sampling estimate of the normalizing constant for each conditional in an autoregressive decomposition. The Autoregressive Energy Machine achieves state-of-the-art performance on a suite of density-estimation tasks.

READ FULL TEXT

page 1

page 5

page 8

research
05/19/2017

Masked Autoregressive Flow for Density Estimation

Autoregressive models are among the best performing neural density estim...
research
05/02/2020

ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation

We propose to train a non-autoregressive machine translation model to mi...
research
01/30/2018

Transformation Autoregressive Networks

The fundamental task of general density estimation has been of keen inte...
research
10/17/2019

Autoregressive Models: What Are They Good For?

Autoregressive (AR) models have become a popular tool for unsupervised l...
research
02/08/2021

Arbitrary Conditional Distributions with Energy

Modeling distributions of covariates, or density estimation, is a core c...
research
07/10/2020

Variable Skipping for Autoregressive Range Density Estimation

Deep autoregressive models compute point likelihood estimates of individ...
research
01/30/2020

Learning Discrete Distributions by Dequantization

Media is generally stored digitally and is therefore discrete. Many succ...

Please sign up or login with your details

Forgot password? Click here to reset