  As various post hoc explanation methods are increasingly being leveraged to
explain complex models in high-stakes settings, it becomes critical to develop
a deeper understanding of if and when the explanations output by these methods
disagree with each other, and how such disagreements are resolved in practice.
However, there is little to no research that provides answers to these critical
questions. In this work, we introduce and study the disagreement problem in
explainable machine learning. More specifically, we formalize the notion of
disagreement between explanations, analyze how often such disagreements occur
in practice, and how practitioners resolve these disagreements. We first
conduct interviews with data scientists to understand what constitutes
disagreement between explanations generated by different methods for the same
model prediction and introduce a novel quantitative framework to formalize this
understanding. We then leverage this framework to carry out a rigorous
empirical analysis with four real-world datasets, six state-of-the-art post hoc
explanation methods, and six different predictive models, to measure the extent
of disagreement between the explanations generated by various popular
explanation methods. In addition, we carry out an online user study with data
scientists to understand how they resolve the aforementioned disagreements. Our
results indicate that (1) state-of-the-art explanation methods often disagree
in terms of the explanations they output, and (2) machine learning
practitioners often employ ad hoc heuristics when resolving such disagreements.
These findings suggest that practitioners may be relying on misleading
explanations when making consequential decisions. They also underscore the
importance of developing principled frameworks for effectively evaluating and
comparing explanations output by various explanation techniques.
