  Existing language models excel at writing from scratch, but many real-world
scenarios require rewriting an existing document to fit a set of constraints.
Although sentence-level rewriting has been fairly well-studied, little work has
addressed the challenge of rewriting an entire document coherently. In this
work, we introduce the task of document-level targeted content transfer and
address it in the recipe domain, with a recipe as the document and a dietary
restriction (such as vegan or dairy-free) as the targeted constraint. We
propose a novel model for this task based on the generative pre-trained
language model (GPT-2) and train on a large number of roughly-aligned recipe
pairs (https://github.com/microsoft/document-level-targeted-content-transfer).
Both automatic and human evaluations show that our model out-performs existing
methods by generating coherent and diverse rewrites that obey the constraint
while remaining close to the original document. Finally, we analyze our model's
rewrites to assess progress toward the goal of making language generation more
attuned to constraints that are substantive rather than stylistic.
