Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting

10/14/2020
by   Adam D. Cobb, et al.
0

Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) approach that exhibits favourable exploration properties in high-dimensional models such as neural networks. Unfortunately, HMC has limited use in large-data regimes and little work has explored suitable approaches that aim to preserve the entire Hamiltonian. In our work, we introduce a new symmetric integration scheme for split HMC that does not rely on stochastic gradients. We show that our new formulation is more efficient than previous approaches and is easy to implement with a single GPU. As a result, we are able to perform full HMC over common deep learning architectures using entire data sets. In addition, when we compare with stochastic gradient MCMC, we show that our method achieves better performance in both accuracy and uncertainty quantification. Our approach demonstrates HMC as a feasible option when considering inference schemes for large-scale machine learning problems.

READ FULL TEXT
research
09/14/2016

Relativistic Monte Carlo

Hamiltonian Monte Carlo (HMC) is a popular Markov chain Monte Carlo (MCM...
research
03/29/2018

Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference

Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics a...
research
03/06/2015

Hamiltonian ABC

Approximate Bayesian computation (ABC) is a powerful and elegant framewo...
research
10/24/2022

On the optimization and pruning for Bayesian deep learning

The goal of Bayesian deep learning is to provide uncertainty quantificat...
research
04/27/2015

Fast Sampling for Bayesian Max-Margin Models

Bayesian max-margin models have shown superiority in various practical a...
research
02/02/2022

Gradient estimators for normalising flows

Recently a machine learning approach to Monte-Carlo simulations called N...
research
02/10/2016

Stochastic Quasi-Newton Langevin Monte Carlo

Recently, Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods...

Please sign up or login with your details

Forgot password? Click here to reset