  Recent work has shown that energy-based language modeling is an effective
framework for controllable text generation because it enables flexible
integration of arbitrary discriminators. However, because energy-based LMs are
globally normalized, approximate techniques like Metropolis-Hastings (MH) are
required for inference. Past work has largely explored simple proposal
distributions that modify a single token at a time, like in Gibbs sampling. In
this paper, we develop a novel MH sampler that, in contrast, proposes re-writes
of the entire sequence in each step via iterative prompting of a large language
model. Our new sampler (a) allows for more efficient and accurate sampling from
a target distribution and (b) allows generation length to be determined through
the sampling procedure rather than fixed in advance, as past work has required.
We perform experiments on two controlled generation tasks, showing both
downstream performance gains and more accurate target distribution sampling in
comparison with single-token proposal techniques.
