  Neural machine translation (NMT) systems aim to map text from one language
into another. While there are a wide variety of applications of NMT, one of the
most important is translation of natural language. A distinguishing factor of
natural language is that words are typically ordered according to the rules of
the grammar of a given language. Although many advances have been made in
developing NMT systems for translating natural language, little research has
been done on understanding how the word ordering of and lexical similarity
between the source and target language affect translation performance. Here, we
investigate these relationships on a variety of low-resource language pairs
from the OpenSubtitles2016 database, where the source language is English, and
find that the more similar the target language is to English, the greater the
translation performance. In addition, we study the impact of providing NMT
models with part of speech of words (POS) in the English sequence and find
that, for Transformer-based models, the more dissimilar the target language is
from English, the greater the benefit provided by POS.
