A Triangular Network For Density Estimation

04/30/2020
by   Xi-Lin Li, et al.
0

In this paper, triangular networks refer to feedforward neural networks with triangular block matrices as their connection weights, and they are studied for density estimation. A special two layer triangular monotonic neural network unit is designed and shown to be universal approximator for invertible mappings with triangular Jacobians based on the simple observation that positively weighted sum of monotonically increasing functions is still monotonic. Then, deep invertible neural networks consisting of stacked such monotonic triangular network units and permutations are proposed as universal density estimators. Our method is most closely related to neural autoregressive density estimations, especially the block neural autoregressive flow. But, unlike many autoregressive models, our designs are highly modular, parameter economy, computationally efficient, and applicable to density estimation of data with high dimensions. Experimental results on image density estimation benchmarks are reported for performance comparisons.

READ FULL TEXT

page 7

page 11

research
05/19/2017

Masked Autoregressive Flow for Density Estimation

Autoregressive models are among the best performing neural density estim...
research
04/09/2019

Block Neural Autoregressive Flow

Normalising flows (NFS) map two density functions via a differentiable b...
research
11/02/2018

Neural Likelihoods via Cumulative Distribution Functions

We leverage neural networks as universal approximators of monotonic func...
research
08/14/2019

Unconstrained Monotonic Neural Networks

Monotonic neural networks have recently been proposed as a way to define...
research
07/10/2020

Variable Skipping for Autoregressive Range Density Estimation

Deep autoregressive models compute point likelihood estimates of individ...
research
07/04/2018

AND: Autoregressive Novelty Detectors

We propose an unsupervised model for novelty detection. The subject is t...
research
10/17/2021

PixelPyramids: Exact Inference Models from Lossless Image Pyramids

Autoregressive models are a class of exact inference approaches with hig...

Please sign up or login with your details

Forgot password? Click here to reset