  We introduce the Hamming Ball Sampler, a novel Markov Chain Monte Carlo
algorithm, for efficient inference in statistical models involving
high-dimensional discrete state spaces. The sampling scheme uses an auxiliary
variable construction that adaptively truncates the model space allowing
iterative exploration of the full model space in polynomial time. The approach
generalizes conventional Gibbs sampling schemes for discrete spaces and can be
considered as a Big Data-enabled MCMC algorithm that provides an intuitive
means for user-controlled balance between statistical efficiency and
computational tractability. We illustrate the generic utility of our sampling
algorithm through application to a range of statistical models.
