AdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation

02/01/2022
by   Emma R. Cobian, et al.
0

Approximating probability distributions can be a challenging task, particularly when they are supported over regions of high geometrical complexity or exhibit multiple modes. Annealing can be used to facilitate this task which is often combined with constant a priori selected increments in inverse temperature. However, using constant increments limit the computational efficiency due to the inability to adapt to situations where smooth changes in the annealed density could be handled equally well with larger increments. We introduce AdaAnn, an adaptive annealing scheduler that automatically adjusts the temperature increments based on the expected change in the Kullback-Leibler divergence between two distributions with a sufficiently close annealing temperature. AdaAnn is easy to implement and can be integrated into existing sampling approaches such as normalizing flows for variational inference and Markov chain Monte Carlo. We demonstrate the computational efficiency of the AdaAnn scheduler for variational inference with normalizing flows on a number of examples, including density approximation and parameter estimation for dynamical systems.

READ FULL TEXT
research
11/07/2014

Variational Tempering

Variational inference (VI) combined with data subsampling enables approx...
research
01/31/2022

Continual Repeated Annealed Flow Transport Monte Carlo

We propose Continual Repeated Annealed Flow Transport Monte Carlo (CRAFT...
research
07/10/2023

LINFA: a Python library for variational inference with normalizing flow and annealing

Variational inference is an increasingly popular method in statistics an...
research
09/15/2022

Rho-Tau Bregman Information and the Geometry of Annealing Paths

Markov Chain Monte Carlo methods for sampling from complex distributions...
research
05/21/2021

Geometric variational inference

Efficiently accessing the information contained in non-linear and high d...
research
06/01/2023

On the Convergence of Coordinate Ascent Variational Inference

As a computational alternative to Markov chain Monte Carlo approaches, v...

Please sign up or login with your details

Forgot password? Click here to reset