Normalizing Flows Across Dimensions

06/23/2020
by   Edmond Cunningham, et al.
18

Real-world data with underlying structure, such as pictures of faces, are hypothesized to lie on a low-dimensional manifold. This manifold hypothesis has motivated state-of-the-art generative algorithms that learn low-dimensional data representations. Unfortunately, a popular generative model, normalizing flows, cannot take advantage of this. Normalizing flows are based on successive variable transformations that are, by design, incapable of learning lower-dimensional representations. In this paper we introduce noisy injective flows (NIF), a generalization of normalizing flows that can go across dimensions. NIF explicitly map the latent space to a learnable manifold in a high-dimensional data space using injective transformations. We further employ an additive noise model to account for deviations from the manifold and identify a stochastic inverse of the generative process. Empirically, we demonstrate that a simple application of our method to existing flow architectures can significantly improve sample quality and yield separable data embeddings.

READ FULL TEXT

page 2

page 6

page 7

page 16

page 17

page 18

page 19

page 20

research
06/18/2020

Variational Autoencoder with Learned Latent Structure

The manifold hypothesis states that high-dimensional data can be modeled...
research
08/26/2023

Out-of-distribution detection using normalizing flows on the data manifold

A common approach for out-of-distribution detection involves estimating ...
research
07/11/2022

Matching Normalizing Flows and Probability Paths on Manifolds

Continuous Normalizing Flows (CNFs) are a class of generative models tha...
research
10/25/2017

Relative Transfer Function Inverse Regression from Low Dimensional Manifold

In room acoustic environments, the Relative Transfer Functions (RTFs) ar...
research
06/15/2020

Ordering Dimensions with Nested Dropout Normalizing Flows

The latent space of normalizing flows must be of the same dimensionality...
research
07/06/2022

The Union of Manifolds Hypothesis and its Implications for Deep Generative Modelling

Deep learning has had tremendous success at learning low-dimensional rep...
research
01/07/2009

A Theoretical Analysis of Joint Manifolds

The emergence of low-cost sensor architectures for diverse modalities ha...

Please sign up or login with your details

Forgot password? Click here to reset