  Deep Learning (DL) is rapidly maturing to the point that it can be used in
safety- and security-crucial applications. However, adversarial samples, which
are undetectable to the human eye, pose a serious threat that can cause the
model to misbehave and compromise the performance of such applications.
Addressing the robustness of DL models has become crucial to understanding and
defending against adversarial attacks. In this study, we perform comprehensive
experiments to examine the effect of adversarial attacks and defenses on
various model architectures across well-known datasets. Our research focuses on
black-box attacks such as SimBA, HopSkipJump, MGAAttack, and boundary attacks,
as well as preprocessor-based defensive mechanisms, including bits squeezing,
median smoothing, and JPEG filter. Experimenting with various models, our
results demonstrate that the level of noise needed for the attack increases as
the number of layers increases. Moreover, the attack success rate decreases as
the number of layers increases. This indicates that model complexity and
robustness have a significant relationship. Investigating the diversity and
robustness relationship, our experiments with diverse models show that having a
large number of parameters does not imply higher robustness. Our experiments
extend to show the effects of the training dataset on model robustness. Using
various datasets such as ImageNet-1000, CIFAR-100, and CIFAR-10 are used to
evaluate the black-box attacks. Considering the multiple dimensions of our
analysis, e.g., model complexity and training dataset, we examined the behavior
of black-box attacks when models apply defenses. Our results show that applying
defense strategies can significantly reduce attack effectiveness. This research
provides in-depth analysis and insight into the robustness of DL models against
various attacks, and defenses.
