Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators

06/20/2020
by   Takeshi Teshima, et al.
0

Invertible neural networks based on coupling flows (CF-INNs) have various machine learning applications such as image synthesis and representation learning. However, their desirable characteristics such as analytic invertibility come at the cost of restricting the functional forms. This poses a question on their representation power: are CF-INNs universal approximators for invertible functions? Without a universality, there could be a well-behaved invertible transformation that the CF-INN can never approximate, hence it would render the model class unreliable. We answer this question by showing a convenient criterion: a CF-INN is universal if its layers contain affine coupling and invertible linear functions as special cases. As its corollary, we can affirmatively resolve a previously unsolved problem: whether normalizing flow models based on affine coupling can be universal distributional approximators. In the course of proving the universality, we prove a general theorem to show the equivalence of the universality for certain diffeomorphism classes, a theoretical insight that is of interest by itself.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/15/2022

Universal approximation property of invertible neural networks

Invertible neural networks (INNs) are neural network architectures with ...
02/07/2022

Universality of parametric Coupling Flows over parametric diffeomorphisms

Invertible neural networks based on Coupling Flows CFlows) have various ...
07/07/2021

Universal Approximation for Log-concave Distributions using Well-conditioned Normalizing Flows

Normalizing flows are a widely used class of latent-variable generative ...
06/01/2020

You say Normalizing Flows I see Bayesian Networks

Normalizing flows have emerged as an important family of deep neural net...
10/02/2020

Representational aspects of depth and conditioning in normalizing flows

Normalizing flows are among the most popular paradigms in generative mod...
08/05/2019

On the Relationship Between Coupling and Refactoring: An Empirical Viewpoint

[Background] Refactoring has matured over the past twenty years to becom...
01/17/2022

Parametrized Convex Universal Approximators for Decision-Making Problems

Parametrized max-affine (PMA) and parametrized log-sum-exp (PLSE) networ...