  While data-driven predictive models are a strictly technological construct,
they may operate within a social context in which benign engineering choices
entail implicit, indirect and unexpected real-life consequences. Fairness of
such systems -- pertaining both to individuals and groups -- is one relevant
consideration in this space; algorithms can discriminate people across various
protected characteristics regardless of whether these properties are included
in the data or discernible through proxy variables. To date, this notion has
predominantly been studied for a fixed model, often under different
classification thresholds, striving to identify and eradicate undesirable,
discriminative and possibly unlawful aspects of its operation. Here, we
backtrack on this fixed model assumption to propose and explore a novel
definition of cross-model fairness where individuals can be harmed when one
predictor is chosen ad hoc from a group of equally well performing models,
i.e., in view of utility-based model multiplicity. Since a person may be
classified differently across models that are otherwise considered equivalent,
this individual could argue for a predictor granting them the most favourable
outcome, employing which may have adverse effects on other people. We introduce
this scenario with a two-dimensional example and linear classification; then,
we present a comprehensive empirical study based on real-life predictive models
and data sets that are popular with the algorithmic fairness community;
finally, we investigate analytical properties of cross-model fairness and its
ramifications in a broader context. Our findings suggest that such unfairness
can be readily found in real life and it may be difficult to mitigate by
technical means alone as doing so is likely to degrade predictive performance.
