Implicit Normalizing Flows

03/17/2021
βˆ™
by   Cheng Lu, et al.
βˆ™
23
βˆ™

Normalizing flows define a probability distribution by an explicit invertible transformation 𝐳=f(𝐱). In this work, we present implicit normalizing flows (ImpFlows), which generalize normalizing flows by allowing the mapping to be implicitly defined by the roots of an equation F(𝐳, 𝐱)= 0. ImpFlows build on residual flows (ResFlows) with a proper balance between expressiveness and tractability. Through theoretical analysis, we show that the function space of ImpFlow is strictly richer than that of ResFlows. Furthermore, for any ResFlow with a fixed number of blocks, there exists some function that ResFlow has a non-negligible approximation error. However, the function is exactly representable by a single-block ImpFlow. We propose a scalable algorithm to train and draw samples from ImpFlows. Empirically, we evaluate ImpFlow on several classification and density modeling tasks, and ImpFlow outperforms ResFlow with a comparable amount of parameters on all the benchmarks.

READ FULL TEXT

page 4

page 5

page 6

page 7

page 10

page 11

page 18

page 23

research
βˆ™ 10/02/2019

Equivariant Flows: sampling configurations for multi-body systems with symmetric energies

Flows are exact-likelihood generative neural networks that transform sam...
research
βˆ™ 03/10/2021

Universal Approximation of Residual Flows in Maximum Mean Discrepancy

Normalizing flows are a class of flexible deep generative models that of...
research
βˆ™ 04/23/2022

Graphical Residual Flows

Graphical flows add further structure to normalizing flows by encoding n...
research
βˆ™ 09/16/2020

Quasi-Autoregressive Residual (QuAR) Flows

Normalizing Flows are a powerful technique for learning and modeling pro...
research
βˆ™ 03/01/2023

Predictive Flows for Faster Ford-Fulkerson

Recent work has shown that leveraging learned predictions can improve th...
research
βˆ™ 10/04/2021

Implicit Riemannian Concave Potential Maps

We are interested in the challenging problem of modelling densities on R...
research
βˆ™ 10/05/2020

i-DenseNets

We introduce Invertible Dense Networks (i-DenseNets), a more parameter e...

Please sign up or login with your details

Forgot password? Click here to reset