  Auto-regressive sequence-to-sequence models with attention mechanism have
achieved state-of-the-art performance in many tasks such as machine translation
and speech synthesis. These models can be difficult to train. The standard
approach, teacher forcing, guides a model with reference output history during
training. The problem is that the model is unlikely to recover from its
mistakes during inference, where the reference output is replaced by generated
output. Several approaches deal with this problem, largely by guiding the model
with generated output history. To make training stable, these approaches often
require a heuristic schedule or an auxiliary classifier. This paper introduces
attention forcing, which guides the model with generated output history and
reference attention. This approach can train the model to recover from its
mistakes, in a stable fashion, without the need for a schedule or a classifier.
In addition, it allows the model to generate output sequences aligned with the
references, which can be important for cascaded systems like many speech
synthesis systems. Experiments on speech synthesis show that attention forcing
yields significant performance gain. Experiments on machine translation show
that for tasks where various re-orderings of the output are valid, guiding the
model with generated output history is challenging, while guiding the model
with reference attention is beneficial.
