  Post hoc explanations have emerged as a way to improve user trust in machine
learning models by providing insight into model decision-making. However,
explanations tend to be evaluated based on their alignment with prior knowledge
while the faithfulness of an explanation with respect to the model, a
fundamental criterion, is often overlooked. Furthermore, the effect of
explanation faithfulness and alignment on user trust and whether this effect
differs among laypeople and domain experts is unclear. To investigate these
questions, we conduct a user study with computer science students and doctors
in three domain areas, controlling the laypeople and domain expert groups in
each setting. The results indicate that laypeople base their trust in
explanations on explanation faithfulness while domain experts base theirs on
explanation alignment. To our knowledge, this work is the first to show that
(1) different factors affect laypeople and domain experts' trust in post hoc
explanations and (2) domain experts are subject to specific biases due to their
expertise when interpreting post hoc explanations. By uncovering this
phenomenon and exposing this cognitive bias, this work motivates the need to
educate end users about how to properly interpret explanations and overcome
their own cognitive biases, and motivates the development of simple and
interpretable faithfulness metrics for end users. This research is particularly
important and timely as post hoc explanations are increasingly being used in
high-stakes, real-world settings such as medicine.
