Jacobian Determinant of Normalizing Flows

02/12/2021
by   Huadong Liao, et al.
24

Normalizing flows learn a diffeomorphic mapping between the target and base distribution, while the Jacobian determinant of that mapping forms another real-valued function. In this paper, we show that the Jacobian determinant mapping is unique for the given distributions, hence the likelihood objective of flows has a unique global optimum. In particular, the likelihood for a class of flows is explicitly expressed by the eigenvalues of the auto-correlation matrix of individual data point, and independent of the parameterization of neural network, which provides a theoretical optimal value of likelihood objective and relates to probabilistic PCA. Additionally, Jacobian determinant is a measure of local volume change and is maximized when MLE is used for optimization. To stabilize normalizing flows training, it is required to maintain a balance between the expansiveness and contraction of volume, meaning Lipschitz constraint on the diffeomorphic mapping and its inverse. With these theoretical results, several principles of designing normalizing flow were proposed. And numerical experiments on highdimensional datasets (such as CelebA-HQ 1024x1024) were conducted to show the improved stability of training.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

page 8

page 13

page 14

03/30/2021

iVPF: Numerical Invertible Volume Preserving Flow for Efficient Lossless Compression

It is nontrivial to store rapidly growing big data nowadays, which deman...
07/15/2021

Copula-Based Normalizing Flows

Normalizing flows, which learn a distribution by transforming the data t...
11/29/2019

Learning Likelihoods with Conditional Normalizing Flows

Normalizing Flows (NFs) are able to model complicated distributions p(y)...
12/31/2021

Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates

Triangular flows, also known as Knöthe-Rosenblatt measure couplings, com...
06/11/2020

Robust model training and generalisation with Studentising flows

Normalising flows are tractable probabilistic models that leverage the p...
10/15/2021

Structured Nonnegative Matrix Factorization for Traffic Flow Estimation of Large Cloud Networks

Network traffic matrix estimation is an ill-posed linear inverse problem...
10/05/2020

i-DenseNets

We introduce Invertible Dense Networks (i-DenseNets), a more parameter e...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.