Log In Sign Up

Differentially Private Hamiltonian Monte Carlo

by   Ossi Räisä, et al.

Markov chain Monte Carlo (MCMC) algorithms have long been the main workhorses of Bayesian inference. Among them, Hamiltonian Monte Carlo (HMC) has recently become very popular due to its efficiency resulting from effective use of the gradients of the target distribution. In privacy-preserving machine learning, differential privacy (DP) has become the gold standard in ensuring that the privacy of data subjects is not violated. Existing DP MCMC algorithms either use random-walk proposals, or do not use the Metropolis–Hastings (MH) acceptance test to ensure convergence without decreasing their step size to zero. We present a DP variant of HMC using the MH acceptance test that builds on a recently proposed DP MCMC algorithm called the penalty algorithm, and adds noise to the gradient evaluations of HMC. We prove that the resulting algorithm converges to the correct distribution, and is ergodic. We compare DP-HMC with the existing penalty, DP-SGLD and DP-SGNHT algorithms, and find that DP-HMC has better or equal performance than the penalty algorithm, and performs more consistently than DP-SGLD or DP-SGNHT.


page 1

page 2

page 3

page 4


Differentially Private Markov Chain Monte Carlo

Recent developments in differentially private (DP) machine learning and ...

Differentially Private Kernel Inducing Points (DP-KIP) for Privacy-preserving Data Distillation

While it is tempting to believe that data distillation preserves privacy...

Bayesian structure learning using dynamic programming and MCMC

MCMC methods for sampling from the space of DAGs can mix poorly due to t...

Exact Privacy Guarantees for Markov Chain Implementations of the Exponential Mechanism with Artificial Atoms

Implementations of the exponential mechanism in differential privacy oft...

Data Subsampling for Bayesian Neural Networks

Markov Chain Monte Carlo (MCMC) algorithms do not scale well for large d...

Differentially Private Bayesian Learning on Distributed Data

Many applications of machine learning, for example in health care, would...

Faster Hamiltonian Monte Carlo by Learning Leapfrog Scale

Hamiltonian Monte Carlo samplers have become standard algorithms for MCM...