  Considerable efforts to measure and mitigate gender bias in recent years have
led to the introduction of an abundance of tasks, datasets, and metrics used in
this vein. In this position paper, we assess the current paradigm of gender
bias evaluation and identify several flaws in it. First, we highlight the
importance of extrinsic bias metrics that measure how a model's performance on
some task is affected by gender, as opposed to intrinsic evaluations of model
representations, which are less strongly connected to specific harms to people
interacting with systems. We find that only a few extrinsic metrics are
measured in most studies, although more can be measured. Second, we find that
datasets and metrics are often coupled, and discuss how their coupling hinders
the ability to obtain reliable conclusions, and how one may decouple them. We
then investigate how the choice of the dataset and its composition, as well as
the choice of the metric, affect bias measurement, finding significant
variations across each of them. Finally, we propose several guidelines for more
reliable gender bias evaluation.
