  Zigzag and other piecewise deterministic Markov process samplers have
attracted significant interest for their non-reversibility and other appealing
properties for Bayesian posterior computation. Hamiltonian Monte Carlo is
another state-of-the-art sampler, exploiting fictitious momentum to guide
Markov chains through complex target distributions. We establish an important
connection between the zigzag sampler and a variant of Hamiltonian Monte Carlo
based on Laplace-distributed momentum. The position and velocity component of
the corresponding Hamiltonian dynamics travels along a zigzag path paralleling
the Markovian zigzag process; however, the dynamics is non-Markovian in this
position-velocity space as the momentum component encodes non-immediate pasts.
This information is partially lost during a momentum refreshment step, in which
we preserve its direction but re-sample magnitude. In the limit of increasingly
frequent momentum refreshments, we prove that Hamiltonian zigzag converges
strongly to its Markovian counterpart. This theoretical insight suggests that,
when retaining full momentum information, Hamiltonian zigzag can better explore
target distributions with highly correlated parameters by suppressing the
diffusive behavior of Markovian zigzag. We corroborate this intuition by
comparing performance of the two zigzag cousins on high-dimensional truncated
multivariate Gaussians, including a 11,235-dimensional target arising from a
Bayesian phylogenetic multivariate probit modeling of HIV virus data.
