  In text generation, models that generate text from scratch one token at a
time are currently the dominant paradigm. Despite being performant, these
models lack the ability to revise existing text, which limits their usability
in many practical scenarios. We look to address this, with DiffusER (Diffusion
via Edit-based Reconstruction), a new edit-based generative model for text
based on denoising diffusion models -- a class of models that use a Markov
chain of denoising steps to incrementally generate data. DiffusER is not only a
strong generative model in general, rivalling autoregressive models on several
tasks spanning machine translation, summarization, and style transfer; it can
also perform other varieties of generation that standard autoregressive models
are not well-suited for. For instance, we demonstrate that DiffusER makes it
possible for a user to condition generation on a prototype, or an incomplete
sequence, and continue revising based on previous edit steps.
