  Document-level machine translation manages to outperform sentence level
models by a small margin, but have failed to be widely adopted. We argue that
previous research did not make a clear use of the global context, and propose a
new document-level NMT framework that deliberately models the local context of
each sentence with the awareness of the global context of the document in both
source and target languages. We specifically design the model to be able to
deal with documents containing any number of sentences, including single
sentences. This unified approach allows our model to be trained elegantly on
standard datasets without needing to train on sentence and document level data
separately. Experimental results demonstrate that our model outperforms
Transformer baselines and previous document-level NMT models with substantial
margins of up to 2.1 BLEU on state-of-the-art baselines. We also provide
analyses which show the benefit of context far beyond the neighboring two or
three sentences, which previous studies have typically incorporated.
