  Unsupervised neural machine translation (UNMT) has recently achieved
remarkable results for several language pairs. However, it can only translate
between a single language pair and cannot produce translation results for
multiple language pairs at the same time. That is, research on multilingual
UNMT has been limited. In this paper, we empirically introduce a simple method
to translate between thirteen languages using a single encoder and a single
decoder, making use of multilingual data to improve UNMT for all language
pairs. On the basis of the empirical findings, we propose two knowledge
distillation methods to further enhance multilingual UNMT performance. Our
experiments on a dataset with English translated to and from twelve other
languages (including three language families and six language branches) show
remarkable results, surpassing strong unsupervised individual baselines while
achieving promising performance between non-English language pairs in zero-shot
translation scenarios and alleviating poor performance in low-resource language
pairs.
