  Plenty of works have brought social biases in language models to attention
and proposed methods to detect such biases. As a result, the literature
contains a great deal of different bias tests and scores, each introduced with
the premise to uncover yet more biases that other scores fail to detect. What
severely lacks in the literature, however, are comparative studies that analyse
such bias scores and help researchers to understand the benefits or limitations
of the existing methods. In this work, we aim to close this gap for cosine
based bias scores. By building on a geometric definition of bias, we propose
requirements for bias scores to be considered meaningful for quantifying
biases. Furthermore, we formally analyze cosine based scores from the
literature with regard to these requirements. We underline these findings with
experiments to show that the bias scores' limitations have an impact in the
application case.
