  Ensuring fairness of machine learning systems is a human-in-the-loop process.
It relies on developers, users, and the general public to identify fairness
problems and make improvements. To facilitate the process we need effective,
unbiased, and user-friendly explanations that people can confidently rely on.
Towards that end, we conducted an empirical study with four types of
programmatically generated explanations to understand how they impact people's
fairness judgments of ML systems. With an experiment involving more than 160
Mechanical Turk workers, we show that: 1) Certain explanations are considered
inherently less fair, while others can enhance people's confidence in the
fairness of the algorithm; 2) Different fairness problems--such as model-wide
fairness issues versus case-specific fairness discrepancies--may be more
effectively exposed through different styles of explanation; 3) Individual
differences, including prior positions and judgment criteria of algorithmic
fairness, impact how people react to different styles of explanation. We
conclude with a discussion on providing personalized and adaptive explanations
to support fairness judgments of ML systems.
