  We revisit the theoretical properties of Hamiltonian stochastic differential
equations (SDES) for Bayesian posterior sampling, and we study the two types of
errors that arise from numerical SDE simulation: the discretization error and
the error due to noisy gradient estimates in the context of data subsampling.
Our main result is a novel analysis for the effect of mini-batches through the
lens of differential operator splitting, revising previous literature results.
The stochastic component of a Hamiltonian SDE is decoupled from the gradient
noise, for which we make no normality assumptions. This leads to the
identification of a convergence bottleneck: when considering mini-batches, the
best achievable error rate is $\mathcal{O}(\eta^2)$, with $\eta$ being the
integrator step size. Our theoretical results are supported by an empirical
study on a variety of regression and classification tasks for Bayesian neural
networks.
