  Deep learning (DL) has shown great success in many human-related tasks, which
has led to its adoption in many computer vision based applications, such as
security surveillance systems, autonomous vehicles and healthcare. Such
safety-critical applications have to draw their path to success deployment once
they have the capability to overcome safety-critical challenges. Among these
challenges are the defense against or/and the detection of the adversarial
examples (AEs). Adversaries can carefully craft small, often imperceptible,
noise called perturbations to be added to the clean image to generate the AE.
The aim of AE is to fool the DL model which makes it a potential risk for DL
applications. Many test-time evasion attacks and countermeasures,i.e., defense
or detection methods, are proposed in the literature. Moreover, few reviews and
surveys were published and theoretically showed the taxonomy of the threats and
the countermeasure methods with little focus in AE detection methods. In this
paper, we focus on image classification task and attempt to provide a survey
for detection methods of test-time evasion attacks on neural network
classifiers. A detailed discussion for such methods is provided with
experimental results for eight state-of-the-art detectors under different
scenarios on four datasets. We also provide potential challenges and future
perspectives for this research direction.
