Why Fairness Cannot Be Automated: Bridging the Gap Between EU
  Non-Discrimination Law and AI
  This article identifies a critical incompatibility between European notions
of discrimination and existing statistical measures of fairness. First, we
review the evidential requirements to bring a claim under EU non-discrimination
law. Due to the disparate nature of algorithmic and human discrimination, the
EU's current requirements are too contextual, reliant on intuition, and open to
judicial interpretation to be automated. Second, we show how the legal
protection offered by non-discrimination law is challenged when AI, not humans,
discriminate. Humans discriminate due to negative attitudes (e.g. stereotypes,
prejudice) and unintentional biases (e.g. organisational practices or
internalised stereotypes) which can act as a signal to victims that
discrimination has occurred. Finally, we examine how existing work on fairness
in machine learning lines up with procedures for assessing cases under EU
non-discrimination law. We propose "conditional demographic disparity" (CDD) as
a standard baseline statistical measurement that aligns with the European Court
of Justice's "gold standard." Establishing a standard set of statistical
evidence for automated discrimination cases can help ensure consistent
procedures for assessment, but not judicial interpretation, of cases involving
AI and automated systems. Through this proposal for procedural regularity in
the identification and assessment of automated discrimination, we clarify how
to build considerations of fairness into automated systems as far as possible
while still respecting and enabling the contextual approach to judicial
interpretation practiced under EU non-discrimination law.
  N.B. Abridged abstract
