  Stochastic differential equations (SDEs) or diffusions are continuous-valued
continuous-time stochastic processes widely used in the applied and
mathematical sciences. Simulating paths from these processes is usually an
intractable problem, and typically involves time-discretization approximations.
We propose an exact Markov chain Monte Carlo sampling algorithm that involves
no such time-discretization error. Our sampler is applicable to the problem of
prior simulation from an SDE, posterior simulation conditioned on noisy
observations, as well as parameter inference given noisy observations. Our work
recasts an existing rejection sampling algorithm for a class of diffusions as a
latent variable model, and then derives an auxiliary variable Gibbs sampling
algorithm that targets the associated joint distribution. At a high level, the
resulting algorithm involves two steps: simulating a random grid of times from
an inhomogeneous Poisson process, and updating the SDE trajectory conditioned
on this grid. Our work allows the vast literature of Monte Carlo sampling
algorithms from the Gaussian process literature to be brought to bear to
applications involving diffusions. We study our method on synthetic and real
datasets, where we demonstrate superior performance over competing methods.
