  Deep neural networks (DNNs) are vulnerable to maliciously generated
adversarial examples. These examples are intentionally designed by making
imperceptible perturbations and often mislead a DNN into making an incorrect
prediction. This phenomenon means that there is significant risk in applying
DNNs to safety-critical applications, such as driverless cars. To address this
issue, we present a visual analytics approach to explain the primary cause of
the wrong predictions introduced by adversarial examples. The key is to analyze
the datapaths of the adversarial examples and compare them with those of the
normal examples. A datapath is a group of critical neurons and their
connections. To this end, we formulate the datapath extraction as a subset
selection problem and approximately solve it based on back-propagation. A
multi-level visualization consisting of a segmented DAG (layer level), an Euler
diagram (feature map level), and a heat map (neuron level), has been designed
to help experts investigate datapaths from the high-level layers to the
detailed neuron activations. Two case studies are conducted that demonstrate
the promise of our approach in support of explaining the working mechanism of
adversarial examples.
