  AI explanations are often mentioned as a way to improve human-AI
decision-making, but empirical studies have not found consistent evidence of
explanations' effectiveness and, on the contrary, suggest that they can
increase overreliance when the AI system is wrong. While many factors may
affect reliance on AI support, one important factor is how decision-makers
reconcile their own intuition -- beliefs or heuristics, based on prior
knowledge, experience, or pattern recognition, used to make judgments -- with
the information provided by the AI system to determine when to override AI
predictions. We conduct a think-aloud, mixed-methods study with two explanation
types (feature- and example-based) for two prediction tasks to explore how
decision-makers' intuition affects their use of AI predictions and
explanations, and ultimately their choice of when to rely on AI. Our results
identify three types of intuition involved in reasoning about AI predictions
and explanations: intuition about the task outcome, features, and AI
limitations. Building on these, we summarize three observed pathways for
decision-makers to apply their own intuition and override AI predictions. We
use these pathways to explain why (1) the feature-based explanations we used
did not improve participants' decision outcomes and increased their
overreliance on AI, and (2) the example-based explanations we used improved
decision-makers' performance over feature-based explanations and helped achieve
complementary human-AI performance. Overall, our work identifies directions for
further development of AI decision-support systems and explanation methods that
help decision-makers effectively apply their intuition to achieve appropriate
reliance on AI.
