  In this paper, we propose Barrier Hamiltonian Monte Carlo (BHMC), a version
of the HMC algorithm which aims at sampling from a Gibbs distribution $\pi$ on
a manifold $\mathrm{M}$, endowed with a Hessian metric $\mathfrak{g}$ derived
from a self-concordant barrier. Our method relies on Hamiltonian dynamics which
comprises $\mathfrak{g}$. Therefore, it incorporates the constraints defining
$\mathrm{M}$ and is able to exploit its underlying geometry. However, the
corresponding Hamiltonian dynamics is defined via non separable Ordinary
Differential Equations (ODEs) in contrast to the Euclidean case. It implies
unavoidable bias in existing generalization of HMC to Riemannian manifolds. In
this paper, we propose a new filter step, called "involution checking step", to
address this problem. This step is implemented in two versions of BHMC, coined
continuous BHMC (c-BHMC) and numerical BHMC (n-BHMC) respectively. Our main
results establish that these two new algorithms generate reversible Markov
chains with respect to $\pi$ and do not suffer from any bias in comparison to
previous implementations. Our conclusions are supported by numerical
experiments where we consider target distributions defined on polytopes.
