  Good quality monolingual word embeddings (MWEs) can be built for languages
which have large amounts of unlabeled text. MWEs can be aligned to bilingual
spaces using only a few thousand word translation pairs. For low resource
languages training MWEs monolingually results in MWEs of poor quality, and thus
poor bilingual word embeddings (BWEs) as well. This paper proposes a new
approach for building BWEs in which the vector space of the high resource
source language is used as a starting point for training an embedding space for
the low resource target language. By using the source vectors as anchors the
vector spaces are automatically aligned during training. We experiment on
English-German, English-Hiligaynon and English-Macedonian. We show that our
approach results not only in improved BWEs and bilingual lexicon induction
performance, but also in improved target language MWE quality as measured using
monolingual word similarity.
