  This work investigates efficient score-based black-box adversarial attacks
with a high Attack Success Rate (ASR) and good generalizability. We design a
novel attack method based on a Disentangled Feature space, called DifAttack,
which differs significantly from the existing ones operating over the entire
feature space. Specifically, DifAttack firstly disentangles an image's latent
feature into an adversarial feature and a visual feature, where the former
dominates the adversarial capability of an image, while the latter largely
determines its visual appearance. We train an autoencoder for the
disentanglement by using pairs of clean images and their Adversarial Examples
(AEs) generated from available surrogate models via white-box attack methods.
Eventually, DifAttack iteratively optimizes the adversarial feature according
to the query feedback from the victim model until a successful AE is generated,
while keeping the visual feature unaltered. In addition, due to the avoidance
of using surrogate models' gradient information when optimizing AEs for
black-box models, our proposed DifAttack inherently possesses better attack
capability in the open-set scenario, where the training dataset of the victim
model is unknown. Extensive experimental results demonstrate that our method
achieves significant improvements in ASR and query efficiency simultaneously,
especially in the targeted attack and open-set scenarios. The code is available
at https://github.com/csjunjun/DifAttack.git.
