  Markov chain Monte Carlo (MCMC) algorithms have long been the main workhorses
of Bayesian inference. Among them, Hamiltonian Monte Carlo (HMC) has recently
become very popular due to its efficiency resulting from effective use of the
gradients of the target distribution. In privacy-preserving machine learning,
differential privacy (DP) has become the gold standard in ensuring that the
privacy of data subjects is not violated. Existing DP MCMC algorithms either
use random-walk proposals, or do not use the Metropolis--Hastings (MH)
acceptance test to ensure convergence without decreasing their step size to
zero. We present a DP variant of HMC using the MH acceptance test that builds
on a recently proposed DP MCMC algorithm called the penalty algorithm, and adds
noise to the gradient evaluations of HMC. We prove that the resulting algorithm
converges to the correct distribution, and is ergodic. We compare DP-HMC with
the existing penalty, DP-SGLD and DP-SGNHT algorithms, and find that DP-HMC has
better or equal performance than the penalty algorithm, and performs more
consistently than DP-SGLD or DP-SGNHT.
