  There has been considerable interest in making Bayesian inference more
scalable. In big data settings, most literature focuses on reducing the
computing time per iteration, with less focused on reducing the number of
iterations needed in Markov chain Monte Carlo (MCMC). This article focuses on
data augmentation MCMC (DA-MCMC), a widely used technique. DA-MCMC samples tend
to become highly autocorrelated in large data samples, due to a miscalibration
problem in which conditional posterior distributions given augmented data are
too concentrated. This makes it necessary to collect very long MCMC paths to
obtain acceptably low MC error. To combat this inefficiency, we propose a
family of calibrated data augmentation algorithms, which appropriately adjust
the variance of conditional posterior distributions. A Metropolis-Hastings step
is used to eliminate bias in the stationary distribution of the resulting
sampler. Compared to existing alternatives, this approach can dramatically
reduce MC error by reducing autocorrelation and increasing the effective number
of DA-MCMC samples per computing time. The approach is simple and applicable to
a broad variety of existing data augmentation algorithms, and we focus on three
popular models: probit, logistic and Poisson log-linear. Dramatic gains in
computational efficiency are shown in applications.
