  Large pretrained language models generate fluent text but are notoriously
hard to controllably sample from. In this work, we study constrained sampling
from such language models: generating text that satisfies user-defined
constraints, while maintaining fluency and the model's performance in a
downstream task. We propose MuCoLa -- a sampling procedure that combines the
log-likelihood of the language model with arbitrary (differentiable)
constraints in a single energy function, and then generates samples in a
non-autoregressive manner. Specifically, it initializes the entire output
sequence with noise and follows a Markov chain defined by Langevin Dynamics
using the gradients of the energy function. We evaluate MuCoLa on text
generation with soft and hard constraints as well as their combinations
obtaining significant improvements over competitive baselines for toxicity
avoidance, sentiment control, and keyword-guided generation.
