Relativistic Monte Carlo

09/14/2016
by   Xiaoyu Lu, et al.
0

Hamiltonian Monte Carlo (HMC) is a popular Markov chain Monte Carlo (MCMC) algorithm that generates proposals for a Metropolis-Hastings algorithm by simulating the dynamics of a Hamiltonian system. However, HMC is sensitive to large time discretizations and performs poorly if there is a mismatch between the spatial geometry of the target distribution and the scales of the momentum distribution. In particular the mass matrix of HMC is hard to tune well. In order to alleviate these problems we propose relativistic Hamiltonian Monte Carlo, a version of HMC based on relativistic dynamics that introduce a maximum velocity on particles. We also derive stochastic gradient versions of the algorithm and show that the resulting algorithms bear interesting relationships to gradient clipping, RMSprop, Adagrad and Adam, popular optimisation methods in deep learning. Based on this, we develop relativistic stochastic gradient descent by taking the zero-temperature limit of relativistic stochastic gradient Hamiltonian Monte Carlo. In experiments we show that the relativistic algorithms perform better than classical Newtonian variants and Adam.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/02/2021

Exact Langevin Dynamics with Stochastic Gradients

Stochastic gradient Markov Chain Monte Carlo algorithms are popular samp...
research
06/03/2015

Parallel Stochastic Gradient Markov Chain Monte Carlo for Matrix Factorisation Models

For large matrix factorisation problems, we develop a distributed Markov...
research
07/01/2020

Decentralized Stochastic Gradient Langevin Dynamics and Hamiltonian Monte Carlo

Stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Ham...
research
10/14/2020

Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splitting

Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) appro...
research
02/17/2014

Stochastic Gradient Hamiltonian Monte Carlo

Hamiltonian Monte Carlo (HMC) sampling methods provide a mechanism for d...
research
10/10/2018

Faster Hamiltonian Monte Carlo by Learning Leapfrog Scale

Hamiltonian Monte Carlo samplers have become standard algorithms for MCM...
research
11/29/2017

Particle Optimization in Stochastic Gradient MCMC

Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has been increasi...

Please sign up or login with your details

Forgot password? Click here to reset