Meta-Learning for Stochastic Gradient MCMC

06/12/2018
by   Wenbo Gong, et al.
0

Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popular for simulating posterior samples in large-scale Bayesian modeling. However, existing SG-MCMC schemes are not tailored to any specific probabilistic model, even a simple modification of the underlying dynamical system requires significant physical intuition. This paper presents the first meta-learning algorithm that allows automated design for the underlying continuous dynamics of an SG-MCMC sampler. The learned sampler generalizes Hamiltonian dynamics with state-dependent drift and diffusion, enabling fast traversal and efficient exploration of neural network energy landscapes. Experiments validate the proposed approach on both Bayesian fully connected neural network and Bayesian recurrent neural network tasks, showing that the learned sampler out-performs generic, hand-designed SG-MCMC algorithms, and generalizes to different datasets and larger architectures.

READ FULL TEXT

page 7

page 17

research
06/15/2015

A Complete Recipe for Stochastic Gradient MCMC

Many recent Markov chain Monte Carlo (MCMC) samplers leverage continuous...
research
06/05/2017

Stochastic Gradient Monomial Gamma Sampler

Recent advances in stochastic gradient techniques have made it possible ...
research
12/02/2016

Asynchronous Stochastic Gradient MCMC with Elastic Coupling

We consider parallel asynchronous Markov Chain Monte Carlo (MCMC) sampli...
research
05/23/2023

Subsampling Error in Stochastic Gradient Langevin Diffusions

The Stochastic Gradient Langevin Dynamics (SGLD) are popularly used to a...
research
02/20/2022

Interacting Contour Stochastic Gradient Langevin Dynamics

We propose an interacting contour stochastic gradient Langevin dynamics ...
research
12/16/2017

How well does your sampler really work?

We present a new data-driven benchmark system to evaluate the performanc...
research
10/21/2019

Aggregated Gradient Langevin Dynamics

In this paper, we explore a general Aggregated Gradient Langevin Dynamic...

Please sign up or login with your details

Forgot password? Click here to reset