  We propose a framework for training non-autoregressive sequence-to-sequence
models for editing tasks, where the original input sequence is iteratively
edited to produce the output. We show that the imitation learning algorithms
designed to train such models for machine translation introduces mismatches
between training and inference that lead to undertraining and poor
generalization in editing scenarios. We address this issue with two
complementary strategies: 1) a roll-in policy that exposes the model to
intermediate training sequences that it is more likely to encounter during
inference, 2) a curriculum that presents easy-to-learn edit operations first,
gradually increasing the difficulty of training samples as the model becomes
competent. We show the efficacy of these strategies on two challenging English
editing tasks: controllable text simplification and abstractive summarization.
Our approach significantly improves output quality on both tasks and controls
output complexity better on the simplification task.
