  Machine learning techniques are increasingly used for high-stakes
decision-making, such as college admissions, loan attribution or recidivism
prediction. Thus, it is crucial to ensure that the models learnt can be audited
or understood by human users, do not create or reproduce discrimination or
bias, and do not leak sensitive information regarding their training data.
Indeed, interpretability, fairness and privacy are key requirements for the
development of responsible machine learning, and all three have been studied
extensively during the last decade. However, they were mainly considered in
isolation, while in practice they interplay with each other, either positively
or negatively. In this Systematization of Knowledge (SoK) paper, we survey the
literature on the interactions between these three desiderata. More precisely,
for each pairwise interaction, we summarize the identified synergies and
tensions. These findings highlight several fundamental theoretical and
empirical conflicts, while also demonstrating that jointly considering these
different requirements is challenging when one aims at preserving a high level
of utility. To solve this issue, we also discuss possible conciliation
mechanisms, showing that a careful design can enable to successfully handle
these different concerns in practice.
