  Deep neural networks obtain state-of-the-art performance on a series of
tasks. However, they are easily fooled by adding a small adversarial
perturbation to input. The perturbation is often human imperceptible on image
data. We observe a significant difference in feature attributions of
adversarially crafted examples from those of original ones. Based on this
observation, we introduce a new framework to detect adversarial examples
through thresholding a scale estimate of feature attribution scores.
Furthermore, we extend our method to include multi-layer feature attributions
in order to tackle the attacks with mixed confidence levels. Through vast
experiments, our method achieves superior performances in distinguishing
adversarial examples from popular attack methods on a variety of real data sets
among state-of-the-art detection methods. In particular, our method is able to
detect adversarial examples of mixed confidence levels, and transfer between
different attacking methods.
