A Variational Perspective on Generative Flow Networks

10/14/2022
by   Heiko Zimmermann, et al.
0

Generative flow networks (GFNs) are a class of models for sequential sampling of composite objects, which approximate a target distribution that is defined in terms of an energy function or a reward. GFNs are typically trained using a flow matching or trajectory balance objective, which matches forward and backward transition models over trajectories. In this work, we define variational objectives for GFNs in terms of the Kullback-Leibler (KL) divergences between the forward and backward distribution. We show that variational inference in GFNs is equivalent to minimizing the trajectory balance objective when sampling trajectories from the forward model. We generalize this approach by optimizing a convex combination of the reverse- and forward KL divergence. This insight suggests variational inference methods can serve as a means to define a more general family of objectives for training generative flow networks, for example by incorporating control variates, which are commonly used in variational inference, to reduce the variance of the gradients of the trajectory balance objective. We evaluate our findings and the performance of the proposed variational objective numerically by comparing it to the trajectory balance objective on two synthetic tasks.

READ FULL TEXT
research
04/10/2023

Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space

Variational inference (VI) seeks to approximate a target distribution π ...
research
06/21/2021

Nested Variational Inference

We develop nested variational inference (NVI), a family of methods that ...
research
09/18/2023

Generalizing Trajectory Retiming to Quadratic Objective Functions

Trajectory retiming is the task of computing a feasible time parameteriz...
research
07/03/2023

Transport, Variational Inference and Diffusions: with Applications to Annealed Flows and Schrödinger Bridges

This paper explores the connections between optimal transport and variat...
research
01/31/2022

Trajectory Balance: Improved Credit Assignment in GFlowNets

Generative Flow Networks (GFlowNets) are a method for learning a stochas...
research
05/11/2023

Towards Understanding and Improving GFlowNet Training

Generative flow networks (GFlowNets) are a family of algorithms that lea...
research
02/03/2023

Better Training of GFlowNets with Local Credit and Incomplete Trajectories

Generative Flow Networks or GFlowNets are related to Monte-Carlo Markov ...

Please sign up or login with your details

Forgot password? Click here to reset