  Deep neural networks are being used increasingly to automate data analysis
and decision making, yet their decision-making process is largely unclear and
is difficult to explain to the end users. In this paper, we address the problem
of Explainable AI for deep neural networks that take images as input and output
a class probability. We propose an approach called RISE that generates an
importance map indicating how salient each pixel is for the model's prediction.
In contrast to white-box approaches that estimate pixel importance using
gradients or other internal network state, RISE works on black-box models. It
estimates importance empirically by probing the model with randomly masked
versions of the input image and obtaining the corresponding outputs. We compare
our approach to state-of-the-art importance extraction methods using both an
automatic deletion/insertion metric and a pointing metric based on
human-annotated object segments. Extensive experiments on several benchmark
datasets show that our approach matches or exceeds the performance of other
methods, including white-box approaches.
  Project page: http://cs-people.bu.edu/vpetsiuk/rise/
