  Bias in Word Embeddings has been a subject of recent interest, along with
efforts for its reduction. Current approaches show promising progress towards
debiasing single bias dimensions such as gender or race. In this paper, we
present a joint multiclass debiasing approach that is capable of debiasing
multiple bias dimensions simultaneously. In that direction, we present two
approaches, HardWEAT and SoftWEAT, that aim to reduce biases by minimizing the
scores of the Word Embeddings Association Test (WEAT). We demonstrate the
viability of our methods by debiasing Word Embeddings on three classes of
biases (religion, gender and race) in three different publicly available word
embeddings and show that our concepts can both reduce or even completely
eliminate bias, while maintaining meaningful relationships between vectors in
word embeddings. Our work strengthens the foundation for more unbiased neural
representations of textual data.
