  Despite the significant advances in deep learning over the past decade, a
major challenge that limits the wide-spread adoption of deep learning has been
their fragility to adversarial attacks. This sensitivity to making erroneous
predictions in the presence of adversarially perturbed data makes deep neural
networks difficult to adopt for certain real-world, mission-critical
applications. While much of the research focus has revolved around adversarial
example creation and adversarial hardening, the area of performance measures
for assessing adversarial robustness is not well explored. Motivated by this,
this study presents the concept of residual error, a new performance measure
for not only assessing the adversarial robustness of a deep neural network at
the individual sample level, but also can be used to differentiate between
adversarial and non-adversarial examples to facilitate for adversarial example
detection. Furthermore, we introduce a hybrid model for approximating the
residual error in a tractable manner. Experimental results using the case of
image classification demonstrates the effectiveness and efficacy of the
proposed residual error metric for assessing several well-known deep neural
network architectures. These results thus illustrate that the proposed measure
could be a useful tool for not only assessing the robustness of deep neural
networks used in mission-critical scenarios, but also in the design of
adversarially robust models.
