  It is well known that Markov chain Monte Carlo (MCMC) methods scale poorly
with dataset size. A popular class of methods for solving this issue is
stochastic gradient MCMC. These methods use a noisy estimate of the gradient of
the log posterior, which reduces the per iteration computational cost of the
algorithm. Despite this, there are a number of results suggesting that
stochastic gradient Langevin dynamics (SGLD), probably the most popular of
these methods, still has computational cost proportional to the dataset size.
We suggest an alternative log posterior gradient estimate for stochastic
gradient MCMC, which uses control variates to reduce the variance. We analyse
SGLD using this gradient estimate, and show that, under log-concavity
assumptions on the target distribution, the computational cost required for a
given level of accuracy is independent of the dataset size. Next we show that a
different control variate technique, known as zero variance control variates
can be applied to SGMCMC algorithms for free. This post-processing step
improves the inference of the algorithm by reducing the variance of the MCMC
output. Zero variance control variates rely on the gradient of the log
posterior; we explore how the variance reduction is affected by replacing this
with the noisy gradient estimate calculated by SGMCMC.
