  Existing machine translation decoding algorithms generate translations in a
strictly monotonic fashion and never revisit previous decisions. As a result,
earlier mistakes cannot be corrected at a later stage. In this paper, we
present a translation scheme that starts from an initial guess and then makes
iterative improvements that may revisit previous decisions. We parameterize our
model as a convolutional neural network that predicts discrete substitutions to
an existing translation based on an attention mechanism over both the source
sentence as well as the current translation output. By making less than one
modification per sentence, we improve the output of a phrase-based translation
system by up to 0.4 BLEU on WMT15 German-English translation.
