  We show how the Hamiltonian Monte Carlo algorithm can sometimes be speeded up
by "splitting" the Hamiltonian in a way that allows much of the movement around
the state space to be done at low computational cost. One context where this is
possible is when the log density of the distribution of interest (the potential
energy function) can be written as the log of a Gaussian density, which is a
quadratic function, plus a slowly varying function. Hamiltonian dynamics for
quadratic energy functions can be analytically solved. With the splitting
technique, only the slowly-varying part of the energy needs to be handled
numerically, and this can be done with a larger stepsize (and hence fewer
steps) than would be necessary with a direct simulation of the dynamics.
Another context where splitting helps is when the most important terms of the
potential energy function and its gradient can be evaluated quickly, with only
a slowly-varying part requiring costly computations. With splitting, the quick
portion can be handled with a small stepsize, while the costly portion uses a
larger stepsize. We show that both of these splitting approaches can reduce the
computational cost of sampling from the posterior distribution for a logistic
regression model, using either a Gaussian approximation centered on the
posterior mode, or a Hamiltonian split into a term that depends on only a small
number of critical cases, and another term that involves the larger number of
cases whose influence on the posterior distribution is small. Supplemental
materials for this paper are available online.
