  Document-level neural machine translation (NMT) has outperformed
sentence-level NMT on a number of datasets. However, document-level NMT is
still not widely adopted in real-world translation systems mainly due to the
lack of large-scale general-domain training data for document-level NMT. We
examine the effectiveness of using Paracrawl for learning document-level
translation. Paracrawl is a large-scale parallel corpus crawled from the
Internet and contains data from various domains. The official Paracrawl corpus
was released as parallel sentences (extracted from parallel webpages) and
therefore previous works only used Paracrawl for learning sentence-level
translation. In this work, we extract parallel paragraphs from Paracrawl
parallel webpages using automatic sentence alignments and we use the extracted
parallel paragraphs as parallel documents for training document-level
translation models. We show that document-level NMT models trained with only
parallel paragraphs from Paracrawl can be used to translate real documents from
TED, News and Europarl, outperforming sentence-level NMT models. We also
perform a targeted pronoun evaluation and show that document-level models
trained with Paracrawl data can help context-aware pronoun translation.
