  Saliency maps can explain how deep neural networks classify images. But are
they actually useful for humans? The present systematic review of 68 user
studies found that while saliency maps can enhance human performance, null
effects or even costs are quite common. To investigate what modulates these
effects, the empirical outcomes were organised along several factors related to
the human tasks, AI performance, XAI methods, images to be classified, human
participants and comparison conditions. In image-focused tasks, benefits were
less common than in AI-focused tasks, but the effects depended on the specific
cognitive requirements. Moreover, benefits were usually restricted to incorrect
AI predictions in AI-focused tasks but to correct ones in image-focused tasks.
XAI-related factors had surprisingly little impact. The evidence was limited
for image- and human-related factors and the effects were highly dependent on
the comparison conditions. These findings may support the design of future user
studies.
