  We introduce a new class of Hamiltonian Monte Carlo (HMC) algorithm called
Conservative Hamiltonian Monte Carlo (CHMC), where energy-preserving
integrators, derived from the Discrete Multiplier Method, are used instead of
symplectic integrators. Due to the volume being no longer preserved under such
a proposal map, a correction involving the determinant of the Jacobian of the
proposal map is introduced within the acceptance probability of HMC. For a
$p$-th order accurate energy-preserving integrator using a time step size
$\tau$, we show that CHMC satisfies stationarity without detailed balance.
Moreover, we show that CHMC satisfies approximate stationarity with an error of
$\mathcal{O}(\tau^{(m+1)p})$ if the determinant of the Jacobian is truncated to
its first $m+1$ terms of its Taylor polynomial in $\tau^p$. We also establish a
lower bound on the acceptance probability of CHMC which depends only on the
desired tolerance $\delta$ for the energy error and approximate determinant. In
particular, a cost-effective and gradient-free version of CHMC is obtained by
approximating the determinant of the Jacobian as unity, leading to an
$\mathcal{O}(\tau^p)$ error to the stationary distribution and a lower bound on
the acceptance probability depending only on $\delta$. Furthermore, numerical
experiments show increased performance in acceptance probability and
convergence to the stationary distribution for the Gradient-free CHMC over HMC
in high dimensional problems.
