  Back-translation (BT) has become one of the de facto components in
unsupervised neural machine translation (UNMT), and it explicitly makes UNMT
have translation ability. However, all the pseudo bi-texts generated by BT are
treated equally as clean data during optimization without considering the
quality diversity, leading to slow convergence and limited translation
performance. To address this problem, we propose a curriculum learning method
to gradually utilize pseudo bi-texts based on their quality from multiple
granularities. Specifically, we first apply cross-lingual word embedding to
calculate the potential translation difficulty (quality) for the monolingual
sentences. Then, the sentences are fed into UNMT from easy to hard batch by
batch. Furthermore, considering the quality of sentences/tokens in a particular
batch are also diverse, we further adopt the model itself to calculate the
fine-grained quality scores, which are served as learning factors to balance
the contributions of different parts when computing loss and encourage the UNMT
model to focus on pseudo data with higher quality. Experimental results on WMT
14 En-Fr, WMT 16 En-De, WMT 16 En-Ro, and LDC En-Zh translation tasks
demonstrate that the proposed method achieves consistent improvements with
faster convergence speed.
