Equivariant Discrete Normalizing Flows

10/16/2021
by   Avishek Joey Bose, et al.
0

At its core, generative modeling seeks to uncover the underlying factors that give rise to observed data that can often be modelled as the natural symmetries that manifest themselves through invariances and equivariances to certain transformations laws. However, current approaches are couched in the formalism of continuous normalizing flows that require the construction of equivariant vector fields – inhibiting their simple application to conventional higher dimensional generative modelling domains like natural images. In this paper we focus on building equivariant normalizing flows using discrete layers. We first theoretically prove the existence of an equivariant map for compact groups whose actions are on compact spaces. We further introduce two new equivariant flows: G-coupling Flows and G-Residual Flows that elevate classical Coupling and Residual Flows with equivariant maps to a prescribed group G. Our construction of G-Residual Flows are also universal, in the sense that we prove an G-equivariant diffeomorphism can be exactly mapped by a G-residual flow. Finally, we complement our theoretical insights with experiments – for the first time – on image datasets like CIFAR-10 and show G-Equivariant Discrete Normalizing flows lead to increased data efficiency, faster convergence, and improved likelihood estimates.

READ FULL TEXT

page 4

page 7

research
09/16/2020

Quasi-Autoregressive Residual (QuAR) Flows

Normalizing Flows are a powerful technique for learning and modeling pro...
research
05/17/2019

Integer Discrete Flows and Lossless Compression

Lossless compression methods shorten the expected representation size of...
research
11/30/2020

General Invertible Transformations for Flow-based Generative Modeling

In this paper, we present a new class of invertible transformations. We ...
research
03/10/2021

Universal Approximation of Residual Flows in Maximum Mean Discrepancy

Normalizing flows are a class of flexible deep generative models that of...
research
10/25/2022

Whitening Convergence Rate of Coupling-based Normalizing Flows

Coupling-based normalizing flows (e.g. RealNVP) are a popular family of ...
research
09/30/2019

Graph Residual Flow for Molecular Graph Generation

Statistical generative models for molecular graphs attract attention fro...
research
10/02/2020

Representational aspects of depth and conditioning in normalizing flows

Normalizing flows are among the most popular paradigms in generative mod...

Please sign up or login with your details

Forgot password? Click here to reset