  Word embeddings have recently been shown to reflect many of the pronounced
societal biases (e.g., gender bias or racial bias). Existing studies are,
however, limited in scope and do not investigate the consistency of biases
across relevant dimensions like embedding models, types of texts, and different
languages. In this work, we present a systematic study of biases encoded in
distributional word vector spaces: we analyze how consistent the bias effects
are across languages, corpora, and embedding models. Furthermore, we analyze
the cross-lingual biases encoded in bilingual embedding spaces, indicative of
the effects of bias transfer encompassed in cross-lingual transfer of NLP
models. Our study yields some unexpected findings, e.g., that biases can be
emphasized or downplayed by different embedding models or that user-generated
content may be less biased than encyclopedic text. We hope our work catalyzes
bias research in NLP and informs the development of bias reduction techniques.
