  Given the severe vulnerability of Deep Neural Networks (DNNs) against
adversarial examples, there is an urgent need for an effective adversarial
attack to identify the deficiencies of DNNs in security-sensitive applications.
As one of the prevalent black-box adversarial attacks, the existing
transfer-based attacks still cannot achieve comparable performance with the
white-box attacks. Among these, input transformation based attacks have shown
remarkable effectiveness in boosting transferability. In this work, we find
that the existing input transformation based attacks transform the input image
globally, resulting in limited diversity of the transformed images. We
postulate that the more diverse transformed images result in better
transferability. Thus, we investigate how to locally apply various
transformations onto the input image to improve such diversity while preserving
the structure of image. To this end, we propose a novel input transformation
based attack, called Structure Invariant Attack (SIA), which applies a random
image transformation onto each image block to craft a set of diverse images for
gradient calculation. Extensive experiments on the standard ImageNet dataset
demonstrate that SIA exhibits much better transferability than the existing
SOTA input transformation based attacks on CNN-based and transformer-based
models, showing its generality and superiority in boosting transferability.
Code is available at https://github.com/xiaosen-wang/SIT.
