  Explaining to users why automated systems make certain mistakes is important
and challenging. Researchers have proposed ways to automatically produce
interpretations for deep neural network models. However, it is unclear how
useful these interpretations are in helping users figure out why they are
getting an error. If an interpretation effectively explains to users how the
underlying deep neural network model works, people who were presented with the
interpretation should be better at predicting the model's outputs than those
who were not. This paper presents an investigation on whether or not showing
machine-generated visual interpretations helps users understand the incorrectly
predicted labels produced by image classifiers. We showed the images and the
correct labels to 150 online crowd workers and asked them to select the
incorrectly predicted labels with or without showing them the machine-generated
visual interpretations. The results demonstrated that displaying the visual
interpretations did not increase, but rather decreased, the average guessing
accuracy by roughly 10%.
