  Although neural machine translation with the encoder-decoder framework has
achieved great success recently, it still suffers drawbacks of forgetting
distant information, which is an inherent disadvantage of recurrent neural
network structure, and disregarding relationship between source words during
encoding step. Whereas in practice, the former information and relationship are
often useful in current step. We target on solving these problems and thus
introduce relation networks to learn better representations of the source. The
relation networks are able to facilitate memorization capability of recurrent
neural network via associating source words with each other, this would also
help retain their relationships. Then the source representations and all the
relations are fed into the attention component together while decoding, with
the main encoder-decoder framework unchanged. Experiments on several datasets
show that our method can improve the translation performance significantly over
the conventional encoder-decoder model and even outperform the approach
involving supervised syntactic knowledge.
