  In the past few years, attention mechanisms have become an indispensable
component of end-to-end neural machine translation models. However, previous
attention models always refer to some source words when predicting a target
word, which contradicts with the fact that some target words have no
corresponding source words. Motivated by this observation, we propose a novel
attention model that has the capability of determining when a decoder should
attend to source words and when it should not. Experimental results on NIST
Chinese-English translation tasks show that the new model achieves an
improvement of 0.8 BLEU score over a state-of-the-art baseline.
