  Unsupervised cross-lingual word embedding (CLWE) methods learn a linear
transformation matrix that maps two monolingual embedding spaces that are
separately trained with monolingual corpora. This method relies on the
assumption that the two embedding spaces are structurally similar, which does
not necessarily hold true in general. In this paper, we argue that using a
pseudo-parallel corpus generated by an unsupervised machine translation model
facilitates the structural similarity of the two embedding spaces and improves
the quality of CLWEs in the unsupervised mapping method. We show that our
approach outperforms other alternative approaches given the same amount of
data, and, through detailed analysis, we show that data augmentation with the
pseudo data from unsupervised machine translation is especially effective for
mapping-based CLWEs because (1) the pseudo data makes the source and target
corpora (partially) parallel; (2) the pseudo data contains information on the
original language that helps to learn similar embedding spaces between the
source and target languages.
