  In this paper, we present a novel Single-class target-specific Adversarial
attack called SingleADV. The goal of SingleADV is to generate a universal
perturbation that deceives the target model into confusing a specific category
of objects with a target category while ensuring highly relevant and accurate
interpretations. The universal perturbation is stochastically and iteratively
optimized by minimizing the adversarial loss that is designed to consider both
the classifier and interpreter costs in targeted and non-targeted categories.
In this optimization framework, ruled by the first- and second-moment
estimations, the desired loss surface promotes high confidence and
interpretation score of adversarial samples. By avoiding unintended
misclassification of samples from other categories, SingleADV enables more
effective targeted attacks on interpretable deep learning systems in both
white-box and black-box scenarios. To evaluate the effectiveness of SingleADV,
we conduct experiments using four different model architectures (ResNet-50,
VGG-16, DenseNet-169, and Inception-V3) coupled with three interpretation
models (CAM, Grad, and MASK). Through extensive empirical evaluation, we
demonstrate that SingleADV effectively deceives the target deep learning models
and their associated interpreters under various conditions and settings. Our
experimental results show that the performance of SingleADV is effective, with
an average fooling ratio of 0.74 and an adversarial confidence level of 0.78 in
generating deceptive adversarial samples. Furthermore, we discuss several
countermeasures against SingleADV, including a transfer-based learning approach
and existing preprocessing defenses.
