  The field explainable artificial intelligence (XAI) has brought about an
arsenal of methods to render Machine Learning (ML) predictions more
interpretable. But how useful explanations provided by transparent ML methods
are for humans remains difficult to assess. Here we investigate the quality of
interpretable computer vision algorithms using techniques from psychophysics.
In crowdsourced annotation tasks we study the impact of different
interpretability approaches on annotation accuracy and task time. We compare
these quality metrics with classical XAI, automated quality metrics. Our
results demonstrate that psychophysical experiments allow for robust quality
assessment of transparency in machine learning. Interestingly the quality
metrics computed without humans in the loop did not provide a consistent
ranking of interpretability methods nor were they representative for how useful
an explanation was for humans. These findings highlight the potential of
methods from classical psychophysics for modern machine learning applications.
We hope that our results provide convincing arguments for evaluating
interpretability in its natural habitat, human-ML interaction, if the goal is
to obtain an authentic assessment of interpretability.
