  Numerous applications in biology, statistics, science, and engineering
require generating samples from high-dimensional probability distributions. In
recent years, the Hamiltonian Monte Carlo (HMC) method has emerged as a
state-of-the-art Markov chain Monte Carlo technique, exploiting the shape of
such high-dimensional target distributions to efficiently generate samples.
Despite its impressive empirical success and increasing popularity, its
wide-scale adoption remains limited due to the high computational cost of
gradient calculation. Moreover, applying this method is impossible when the
gradient of the posterior cannot be computed (for example, with black-box
simulators). To overcome these challenges, we propose a novel two-stage
Hamiltonian Monte Carlo algorithm with a surrogate model. In this
multi-fidelity algorithm, the acceptance probability is computed in the first
stage via a standard HMC proposal using an inexpensive differentiable surrogate
model, and if the proposal is accepted, the posterior is evaluated in the
second stage using the high-fidelity (HF) numerical solver. Splitting the
standard HMC algorithm into these two stages allows for approximating the
gradient of the posterior efficiently, while producing accurate posterior
samples by using HF numerical solvers in the second stage. We demonstrate the
effectiveness of this algorithm for a range of problems, including linear and
nonlinear Bayesian inverse problems with in-silico data and experimental data.
The proposed algorithm is shown to seamlessly integrate with various
low-fidelity and HF models, priors, and datasets. Remarkably, our proposed
method outperforms the traditional HMC algorithm in both computational and
statistical efficiency by several orders of magnitude, all while retaining or
improving the accuracy in computed posterior statistics.
