Fast Mixing of Multi-Scale Langevin Dynamics under the Manifold Hypothesis

06/19/2020
by   Adam Block, et al.
6

Recently, the task of image generation has attracted much attention. In particular, the recent empirical successes of the Markov Chain Monte Carlo (MCMC) technique of Langevin Dynamics have prompted a number of theoretical advances; despite this, several outstanding problems remain. First, the Langevin Dynamics is run in very high dimension on a nonconvex landscape; in the worst case, due to the NP-hardness of nonconvex optimization, it is thought that Langevin Dynamics mixes only in time exponential in the dimension. In this work, we demonstrate how the manifold hypothesis allows for the considerable reduction of mixing time, from exponential in the ambient dimension to depending only on the (much smaller) intrinsic dimension of the data. Second, the high dimension of the sampling space significantly hurts the performance of Langevin Dynamics; we leverage a multi-scale approach to help ameliorate this issue and observe that this multi-resolution algorithm allows for a trade-off between image quality and computational expense in generation.

READ FULL TEXT

page 8

page 20

page 21

page 22

page 23

page 24

page 25

page 26

research
06/19/2020

Fast Mixing of Multi-Scale Langevin Dynamics underthe Manifold Hypothesis

Recently, the task of image generation has attracted much attention. In ...
research
07/01/2018

Heuristic Framework for Multi-Scale Testing of the Multi-Manifold Hypothesis

When analyzing empirical data, we often find that global linear models o...
research
08/28/2022

Metastable Mixing of Markov Chains: Efficiently Sampling Low Temperature Exponential Random Graphs

In this paper we consider the problem of sampling from the low-temperatu...
research
08/02/2021

Asymptotic bias of inexact Markov Chain Monte Carlo methods in high dimension

This paper establishes non-asymptotic bounds on Wasserstein distances be...
research
10/21/2019

Aggregated Gradient Langevin Dynamics

In this paper, we explore a general Aggregated Gradient Langevin Dynamic...
research
12/05/2019

PICAR: An Efficient Extendable Approach for Fitting Hierarchical Spatial Models

Hierarchical spatial models are very flexible and popular for a vast arr...

Please sign up or login with your details

Forgot password? Click here to reset