  A vast number of systems across the world use algorithmic decision making
(ADM) to (partially) automate decisions that have previously been made by
humans. The downstream effects of ADM systems critically depend on the
decisions made during a systems' design, implementation, and evaluation, as
biases in data can be mitigated or reinforced along the modeling pipeline. Many
of these decisions are made implicitly, without knowing exactly how they will
influence the final system. To study this issue, we draw on insights from the
field of psychology and introduce the method of multiverse analysis for
algorithmic fairness. In our proposed method, we turn implicit decisions during
design and evaluation into explicit ones and demonstrate their fairness
implications. By combining decisions, we create a grid of all possible
"universes" of decision combinations. For each of these universes, we compute
metrics of fairness and performance. Using the resulting dataset, one can
investigate the variability and robustness of fairness scores and see how and
which decisions impact fairness. We demonstrate how multiverse analyses can be
used to better understand fairness implications of design and evaluation
decisions using an exemplary case study of predicting public health care
coverage for vulnerable populations. Our results highlight how decisions
regarding the evaluation of a system can lead to vastly different fairness
metrics for the same model. This is problematic, as a nefarious actor could
optimise or "hack" a fairness metric to portray a discriminating model as fair
merely by changing how it is evaluated. We illustrate how a multiverse analysis
can help to address this issue.
