Unconstrained Monotonic Neural Networks

08/14/2019
by   Antoine Wehenkel, et al.
1

Monotonic neural networks have recently been proposed as a way to define invertible transformations. These transformations can be combined into powerful autoregressive flows that have been shown to be universal approximators of continuous probability distributions. Architectures that ensure monotonicity typically enforce constraints on weights and activation functions, which enables invertibility but leads to a cap on the expressiveness of the resulting transformations. In this work, we propose the Unconstrained Monotonic Neural Network (UMNN) architecture based on the insight that a function is monotonic as long as its derivative is strictly positive. In particular, this latter condition can be enforced with a free-form neural network whose only constraint is the positiveness of its output. We evaluate our new invertible building block within a new autoregressive flow (UMNN-MAF) and demonstrate its effectiveness on density estimation experiments. We also illustrate the ability of UMNNs to improve variational inference.

READ FULL TEXT

page 6

page 7

page 13

research
04/03/2018

Neural Autoregressive Flows

Normalizing flows and autoregressive models have been successfully combi...
research
04/30/2020

A Triangular Network For Density Estimation

In this paper, triangular networks refer to feedforward neural networks ...
research
04/09/2019

Block Neural Autoregressive Flow

Normalising flows (NFS) map two density functions via a differentiable b...
research
06/05/2022

AUTM Flow: Atomic Unrestricted Time Machine for Monotonic Normalizing Flows

Nonlinear monotone transformations are used extensively in normalizing f...
research
06/15/2016

Improving Variational Inference with Inverse Autoregressive Flow

The framework of normalizing flows provides a general strategy for flexi...
research
11/02/2018

Neural Likelihoods via Cumulative Distribution Functions

We leverage neural networks as universal approximators of monotonic func...
research
07/14/2023

Expressive Monotonic Neural Networks

The monotonic dependence of the outputs of a neural network on some of i...

Please sign up or login with your details

Forgot password? Click here to reset