  In Transformer-based neural machine translation (NMT), the positional
encoding mechanism helps the self-attention networks to learn the source
representation with order dependency, which makes the Transformer-based NMT
achieve state-of-the-art results for various translation tasks. However,
Transformer-based NMT only adds representations of positions sequentially to
word vectors in the input sentence and does not explicitly consider reordering
information in this sentence. In this paper, we first empirically investigate
the relationship between source reordering information and translation
performance. The empirical findings show that the source input with the target
order learned from the bilingual parallel dataset can substantially improve
translation performance. Thus, we propose a novel reordering method to
explicitly model this reordering information for the Transformer-based NMT. The
empirical results on the WMT14 English-to-German, WAT ASPEC
Japanese-to-English, and WMT17 Chinese-to-English translation tasks show the
effectiveness of the proposed approach.
