  This paper introduces an alternative approach to sampling from autoregressive
models. Autoregressive models are typically sampled sequentially, according to
the transition dynamics defined by the model. Instead, we propose a sampling
procedure that initializes a sequence with white noise and follows a Markov
chain defined by Langevin dynamics on the global log-likelihood of the
sequence. This approach parallelizes the sampling process and generalizes to
conditional sampling. Using an autoregressive model as a Bayesian prior, we can
steer the output of a generative model using a conditional likelihood or
constraints. We apply these techniques to autoregressive models in the visual
and audio domains, with competitive results for audio source separation,
super-resolution, and inpainting.
