OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport

05/29/2020
by   Derek Onken, et al.
15

A normalizing flow is an invertible mapping between an arbitrary probability distribution and a standard normal distribution; it can be used for density estimation and statistical inference. Computing the flow follows the change of variables formula and thus requires invertibility of the mapping and an efficient way to compute the determinant of its Jacobian. To satisfy these requirements, normalizing flows typically consist of carefully chosen components. Continuous normalizing flows (CNFs) are mappings obtained by solving a neural ordinary differential equation (ODE). The neural ODE's dynamics can be chosen almost arbitrarily while ensuring invertibility. Moreover, the log-determinant of the flow's Jacobian can be obtained by integrating the trace of the dynamics' Jacobian along the flow. Our proposed OT-Flow approach tackles two critical computational challenges that limit a more widespread use of CNFs. First, OT-Flow leverages optimal transport (OT) theory to regularize the CNF and enforce straight trajectories that are easier to integrate. Second, OT-Flow features exact trace computation with time complexity equal to trace estimators used in existing CNFs. On five high-dimensional density estimation and generative modeling tasks, OT-Flow performs competitively to a state-of-the-art CNF while on average requiring one-fourth of the number of weights with 17x speedup in training time and 28x speedup in inference.

READ FULL TEXT

page 6

page 7

page 8

page 13

page 16

page 18

page 19

page 20

research
11/30/2022

Taming Hyperparameter Tuning in Continuous Normalizing Flows Using the JKO Scheme

A normalizing flow (NF) is a mapping that transforms a chosen probabilit...
research
08/26/2023

SyMOT-Flow: Learning optimal transport flow for two arbitrary distributions with maximum mean discrepancy

Finding a transformation between two unknown probability distributions f...
research
03/19/2022

TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed

Continuous normalizing flows (CNFs) construct invertible mappings betwee...
research
12/10/2020

Convex Potential Flows: Universal Probability Distributions with Optimal Transport and Convex Optimization

Flow-based models are powerful tools for designing probabilistic models ...
research
09/07/2022

Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

We present rectified flow, a surprisingly simple approach to learning (n...
research
12/08/2020

Accelerating Continuous Normalizing Flow with Trajectory Polynomial Regularization

In this paper, we propose an approach to effectively accelerating the co...
research
09/26/2018

Monge-Ampère Flow for Generative Modeling

We present a deep generative model, named Monge-Ampère flow, which build...

Please sign up or login with your details

Forgot password? Click here to reset