  Unsupervised translation has reached impressive performance on resource-rich
language pairs such as English-French and English-German. However, early
studies have shown that in more realistic settings involving low-resource, rare
languages, unsupervised translation performs poorly, achieving less than 3.0
BLEU. In this work, we show that multilinguality is critical to making
unsupervised systems practical for low-resource settings. In particular, we
present a single model for 5 low-resource languages (Gujarati, Kazakh, Nepali,
Sinhala, and Turkish) to and from English directions, which leverages
monolingual and auxiliary parallel data from other high-resource language pairs
via a three-stage training scheme. We outperform all current state-of-the-art
unsupervised baselines for these languages, achieving gains of up to 14.4 BLEU.
Additionally, we outperform a large collection of supervised WMT submissions
for various language pairs as well as match the performance of the current
state-of-the-art supervised model for Nepali-English. We conduct a series of
ablation studies to establish the robustness of our model under different
degrees of data quality, as well as to analyze the factors which led to the
superior performance of the proposed approach over traditional unsupervised
models.
