Matching Normalizing Flows and Probability Paths on Manifolds

07/11/2022
by   Heli Ben-Hamu, et al.
12

Continuous Normalizing Flows (CNFs) are a class of generative models that transform a prior distribution to a model distribution by solving an ordinary differential equation (ODE). We propose to train CNFs on manifolds by minimizing probability path divergence (PPD), a novel family of divergences between the probability density path generated by the CNF and a target probability density path. PPD is formulated using a logarithmic mass conservation formula which is a linear first order partial differential equation relating the log target probabilities and the CNF's defining vector field. PPD has several key benefits over existing methods: it sidesteps the need to solve an ODE per iteration, readily applies to manifold data, scales to high dimensions, and is compatible with a large family of target paths interpolating pure noise and data in finite time. Theoretically, PPD is shown to bound classical probability divergences. Empirically, we show that CNFs learned by minimizing PPD achieve state-of-the-art results in likelihoods and sample quality on existing low-dimensional manifold benchmarks, and is the first example of a generative model to scale to moderately high dimensional manifolds.

READ FULL TEXT

page 3

page 7

page 9

page 15

research
08/18/2021

Moser Flow: Divergence-based Generative Modeling on Manifolds

We are interested in learning generative models for complex geometries d...
research
06/23/2020

Normalizing Flows Across Dimensions

Real-world data with underlying structure, such as pictures of faces, ar...
research
06/11/2023

On Kinetic Optimal Probability Paths for Generative Models

Recent successful generative models are trained by fitting a neural netw...
research
06/01/2023

A Neural RDE-based model for solving path-dependent PDEs

The concept of the path-dependent partial differential equation (PPDE) w...
research
04/28/2023

Multisample Flow Matching: Straightening Flows with Minibatch Couplings

Simulation-free methods for training continuous-time generative models c...

Please sign up or login with your details

Forgot password? Click here to reset