Principal Manifold Flows

by   Edmond Cunningham, et al.

Normalizing flows map an independent set of latent variables to their samples using a bijective transformation. Despite the exact correspondence between samples and latent variables, their high level relationship is not well understood. In this paper we characterize the geometric structure of flows using principal manifolds and understand the relationship between latent variables and samples using contours. We introduce a novel class of normalizing flows, called principal manifold flows (PF), whose contours are its principal manifolds, and a variant for injective flows (iPF) that is more efficient to train than regular injective flows. PFs can be constructed using any flow architecture, are trained with a regularized maximum likelihood objective and can perform density estimation on all of their principal manifolds. In our experiments we show that PFs and iPFs are able to learn the principal manifolds over a variety of datasets. Additionally, we show that PFs can perform density estimation on data that lie on a manifold with variable dimensionality, which is not possible with existing normalizing flows.


page 10

page 24

page 27


Ordering Dimensions with Nested Dropout Normalizing Flows

The latent space of normalizing flows must be of the same dimensionality...

Lie PCA: Density estimation for symmetric manifolds

We introduce an extension to local principal component analysis for lear...

Principal Boundary on Riemannian Manifolds

We revisit the classification problem and focus on nonlinear methods for...

Manifold Learning Using Kernel Density Estimation and Local Principal Components Analysis

We consider the problem of recovering a d-dimensional manifold M⊂R^n whe...

Nonlinear Isometric Manifold Learning for Injective Normalizing Flows

To model manifold data using normalizing flows, we propose to employ the...

Manifold unwrapping using density ridges

Research on manifold learning within a density ridge estimation framewor...

Horizontal Flows and Manifold Stochastics in Geometric Deep Learning

We introduce two constructions in geometric deep learning for 1) transpo...