Differentially Private Hamiltonian Monte Carlo

06/17/2021
by   Ossi Räisä, et al.
0

Markov chain Monte Carlo (MCMC) algorithms have long been the main workhorses of Bayesian inference. Among them, Hamiltonian Monte Carlo (HMC) has recently become very popular due to its efficiency resulting from effective use of the gradients of the target distribution. In privacy-preserving machine learning, differential privacy (DP) has become the gold standard in ensuring that the privacy of data subjects is not violated. Existing DP MCMC algorithms either use random-walk proposals, or do not use the Metropolis–Hastings (MH) acceptance test to ensure convergence without decreasing their step size to zero. We present a DP variant of HMC using the MH acceptance test that builds on a recently proposed DP MCMC algorithm called the penalty algorithm, and adds noise to the gradient evaluations of HMC. We prove that the resulting algorithm converges to the correct distribution, and is ergodic. We compare DP-HMC with the existing penalty, DP-SGLD and DP-SGNHT algorithms, and find that DP-HMC has better or equal performance than the penalty algorithm, and performs more consistently than DP-SGLD or DP-SGNHT.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/29/2019

Differentially Private Markov Chain Monte Carlo

Recent developments in differentially private (DP) machine learning and ...
research
03/10/2023

DP-Fast MH: Private, Fast, and Accurate Metropolis-Hastings for Large-Scale Bayesian Inference

Bayesian inference provides a principled framework for learning from com...
research
06/20/2012

Bayesian structure learning using dynamic programming and MCMC

MCMC methods for sampling from the space of DAGs can mix poorly due to t...
research
04/03/2022

Exact Privacy Guarantees for Markov Chain Implementations of the Exponential Mechanism with Artificial Atoms

Implementations of the exponential mechanism in differential privacy oft...
research
03/03/2017

Differentially Private Bayesian Learning on Distributed Data

Many applications of machine learning, for example in health care, would...
research
03/01/2023

Two Views of Constrained Differential Privacy: Belief Revision and Update

In this paper, we provide two views of constrained differential private ...
research
01/15/2020

Newtonian Monte Carlo: single-site MCMC meets second-order gradient methods

Single-site Markov Chain Monte Carlo (MCMC) is a variant of MCMC in whic...

Please sign up or login with your details

Forgot password? Click here to reset