Flow Annealed Importance Sampling Bootstrap

08/03/2022
by   Laurence Illing Midgley, et al.
12

Normalizing flows are tractable density models that can approximate complicated target distributions, e.g. Boltzmann distributions of physical systems. However, current methods for training flows either suffer from mode-seeking behavior, use samples from the target generated beforehand by expensive MCMC simulations, or use stochastic losses that have very high variance. To avoid these problems, we augment flows with annealed importance sampling (AIS) and minimize the mass covering α-divergence with α=2, which minimizes importance weight variance. Our method, Flow AIS Bootstrap (FAB), uses AIS to generate samples in regions where the flow is a poor approximation of the target, facilitating the discovery of new modes. We target with AIS the minimum variance distribution for the estimation of the α-divergence via importance sampling. We also use a prioritized buffer to store and reuse AIS samples. These two features significantly improve FAB's performance. We apply FAB to complex multimodal targets and show that we can approximate them very accurately where previous methods fail. To the best of our knowledge, we are the first to learn the Boltzmann distribution of the alanine dipeptide molecule using only the unnormalized target density and without access to samples generated via Molecular Dynamics (MD) simulations: FAB produces better results than training via maximum likelihood on MD samples while using 100 times fewer target evaluations. After reweighting samples with importance weights, we obtain unbiased histograms of dihedral angles that are almost identical to the ground truth ones.

READ FULL TEXT

page 8

page 23

page 26

page 31

page 32

page 33

page 34

page 35

research
11/22/2021

Bootstrap Your Flow

Normalizing flows are flexible, parameterized distributions that can be ...
research
08/20/2023

SE(3) Equivariant Augmented Coupling Flows

Coupling normalizing flows allow for fast sampling and density evaluatio...
research
01/13/2023

Designing losses for data-free training of normalizing flows on Boltzmann distributions

Generating a Boltzmann distribution in high dimension has recently been ...
research
06/30/2023

Learned harmonic mean estimation of the marginal likelihood with normalizing flows

Computing the marginal likelihood (also called the Bayesian model eviden...
research
03/23/2020

Efficient sampling generation from explicit densities via Normalizing Flows

For many applications, such as computing the expected value of different...
research
11/16/2021

Normalizing flows for atomic solids

We present a machine-learning approach, based on normalizing flows, for ...
research
02/27/2023

Detecting and Mitigating Mode-Collapse for Flow-based Sampling of Lattice Field Theories

We study the consequences of mode-collapse of normalizing flows in the c...

Please sign up or login with your details

Forgot password? Click here to reset