Quantum-Inspired Hamiltonian Monte Carlo for Bayesian Sampling

by   Ziming Liu, et al.

Hamiltonian Monte Carlo (HMC) is an efficient Bayesian sampling method that can make distant proposals in the parameter space by simulating a Hamiltonian dynamical system. Despite its popularity in machine learning and data science, HMC is inefficient to sample from spiky and multimodal distributions. Motivated by the energy-time uncertainty relation from quantum mechanics, we propose a Quantum-Inspired Hamiltonian Monte Carlo algorithm (QHMC). This algorithm allows a particle to have a random mass with a probability distribution rather than a fixed mass. We prove the convergence property of QHMC in the spatial domain and in the time sequence. We further show why such a random mass can improve the performance when we sample a broad class of distributions. In order to handle the big training data sets in large-scale machine learning, we develop a stochastic gradient version of QHMC using Nosé-Hoover thermostat called QSGNHT, and we also provide theoretical justifications about its steady-state distributions. Finally in the experiments, we demonstrate the effectiveness of QHMC and QSGNHT on synthetic examples, bridge regression, image denoising and neural network pruning. The proposed QHMC and QSGNHT can indeed achieve much more stable and accurate sampling results on the test cases.


page 5

page 7

page 12

page 16

page 18

page 22

page 24

page 26


Antithetic Riemannian Manifold And Quantum-Inspired Hamiltonian Monte Carlo

Markov Chain Monte Carlo inference of target posterior distributions in ...

Neural Network Gradient Hamiltonian Monte Carlo

Hamiltonian Monte Carlo is a widely used algorithm for sampling from pos...

Predictive Uncertainty in Large Scale Classification using Dropout - Stochastic Gradient Hamiltonian Monte Carlo

Predictive uncertainty is crucial for many computer vision tasks, from i...

Stochastic Gradient Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) sampling methods provide a mechanism for d...

Towards Unifying Hamiltonian Monte Carlo and Slice Sampling

We unify slice sampling and Hamiltonian Monte Carlo (HMC) sampling, demo...

Decentralized Bayesian Learning with Metropolis-Adjusted Hamiltonian Monte Carlo

Federated learning performed by a decentralized networks of agents is be...

A splitting Hamiltonian Monte Carlo method for efficient sampling

We propose a splitting Hamiltonian Monte Carlo (SHMC) algorithm, which c...