  Displaying confidence scores in human-AI interaction has been shown to help
build trust between humans and AI systems. However, most existing research uses
only the confidence score as a form of communication. As confidence scores are
just another model output, users may want to understand why the algorithm is
confident to determine whether to accept the confidence score. In this paper,
we show that counterfactual explanations of confidence scores help study
participants to better understand and better trust a machine learning model's
prediction. We present two methods for understanding model confidence using
counterfactual explanation: (1) based on counterfactual examples; and (2) based
on visualisation of the counterfactual space. Both increase understanding and
trust for study participants over a baseline of no explanation, but qualitative
results show that they are used quite differently, leading to recommendations
of when to use each one and directions of designing better explanations.
