  Neural machine translation models usually adopt the teacher forcing strategy
for training which requires the predicted sequence matches ground truth word by
word and forces the probability of each prediction to approach a 0-1
distribution. However, the strategy casts all the portion of the distribution
to the ground truth word and ignores other words in the target vocabulary even
when the ground truth word cannot dominate the distribution. To address the
problem of teacher forcing, we propose a method to introduce an evaluation
module to guide the distribution of the prediction. The evaluation module
accesses each prediction from the perspectives of fluency and faithfulness to
encourage the model to generate the word which has a fluent connection with its
past and future translation and meanwhile tends to form a translation
equivalent in meaning to the source. The experiments on multiple translation
tasks show that our method can achieve significant improvements over strong
baselines.
