  This study explores the impact of adversarial perturbations on Convolutional
Neural Networks (CNNs) with the aim of enhancing the understanding of their
underlying mechanisms. Despite numerous defense methods proposed in the
literature, there is still an incomplete understanding of this phenomenon.
Instead of treating the entire model as vulnerable, we propose that specific
feature maps learned during training contribute to the overall vulnerability.
To investigate how the hidden representations learned by a CNN affect its
vulnerability, we introduce the Adversarial Intervention framework. Experiments
were conducted on models trained on three well-known computer vision datasets,
subjecting them to attacks of different nature. Our focus centers on the
effects that adversarial perturbations to a model's initial layer have on the
overall behavior of the model. Empirical results revealed compelling insights:
a) perturbing selected channel combinations in shallow layers causes
significant disruptions; b) the channel combinations most responsible for the
disruptions are common among different types of attacks; c) despite shared
vulnerable combinations of channels, different attacks affect hidden
representations with varying magnitudes; d) there exists a positive correlation
between a kernel's magnitude and its vulnerability. In conclusion, this work
introduces a novel framework to study the vulnerability of a CNN model to
adversarial perturbations, revealing insights that contribute to a deeper
understanding of the phenomenon. The identified properties pave the way for the
development of efficient ad-hoc defense mechanisms in future applications.
