Normalizing Flows Across Dimensions

by   Edmond Cunningham, et al.

Real-world data with underlying structure, such as pictures of faces, are hypothesized to lie on a low-dimensional manifold. This manifold hypothesis has motivated state-of-the-art generative algorithms that learn low-dimensional data representations. Unfortunately, a popular generative model, normalizing flows, cannot take advantage of this. Normalizing flows are based on successive variable transformations that are, by design, incapable of learning lower-dimensional representations. In this paper we introduce noisy injective flows (NIF), a generalization of normalizing flows that can go across dimensions. NIF explicitly map the latent space to a learnable manifold in a high-dimensional data space using injective transformations. We further employ an additive noise model to account for deviations from the manifold and identify a stochastic inverse of the generative process. Empirically, we demonstrate that a simple application of our method to existing flow architectures can significantly improve sample quality and yield separable data embeddings.


page 2

page 6

page 7

page 16

page 17

page 18

page 19

page 20


Variational Autoencoder with Learned Latent Structure

The manifold hypothesis states that high-dimensional data can be modeled...

Joint Manifold Learning and Density Estimation Using Normalizing Flows

Based on the manifold hypothesis, real-world data often lie on a low-dim...

Ordering Dimensions with Nested Dropout Normalizing Flows

The latent space of normalizing flows must be of the same dimensionality...

The Union of Manifolds Hypothesis and its Implications for Deep Generative Modelling

Deep learning has had tremendous success at learning low-dimensional rep...

Matching Normalizing Flows and Probability Paths on Manifolds

Continuous Normalizing Flows (CNFs) are a class of generative models tha...

Learning disentangled representations via product manifold projection

We propose a novel approach to disentangle the generative factors of var...

Relative Transfer Function Inverse Regression from Low Dimensional Manifold

In room acoustic environments, the Relative Transfer Functions (RTFs) ar...