  Word embeddings represent words in a numeric space so that semantic relations
between words are represented as distances and directions in the vector space.
Cross-lingual word embeddings transform vector spaces of different languages so
that similar words are aligned. This is done by constructing a mapping between
vector spaces of two languages or learning a joint vector space for multiple
languages. Cross-lingual embeddings can be used to transfer machine learning
models between languages, thereby compensating for insufficient data in
less-resourced languages. We use cross-lingual word embeddings to transfer
machine learning prediction models for Twitter sentiment between 13 languages.
We focus on two transfer mechanisms that recently show superior transfer
performance. The first mechanism uses the trained models whose input is the
joint numerical space for many languages as implemented in the LASER library.
The second mechanism uses large pretrained multilingual BERT language models.
Our experiments show that the transfer of models between similar languages is
sensible, even with no target language data. The performance of cross-lingual
models obtained with the multilingual BERT and LASER library is comparable, and
the differences are language-dependent. The transfer with CroSloEngual BERT,
pretrained on only three languages, is superior on these and some closely
related languages.
