  The evaluation of explainable artificial intelligence is challenging, because
automated and human-centred metrics of explanation quality may diverge. To
clarify their relationship, we investigated whether human and artificial image
classification will benefit from the same visual explanations. In three
experiments, we analysed human reaction times, errors, and subjective ratings
while participants classified image segments. These segments either reflected
human attention (eye movements, manual selections) or the outputs of two
attribution methods explaining a ResNet (Grad-CAM, XRAI). We also had this
model classify the same segments. Humans and the model largely agreed on the
interpretability of attribution methods: Grad-CAM was easily interpretable for
indoor scenes and landscapes, but not for objects, while the reverse pattern
was observed for XRAI. Conversely, human and model performance diverged for
human-generated segments. Our results caution against general statements about
interpretability, as it varies with the explanation method, the explained
images, and the agent interpreting them.
