  Adversarial transferability enables black-box attacks on unknown victim deep
neural networks (DNNs), rendering attacks viable in real-world scenarios.
Current transferable attacks create adversarial perturbation over the entire
image, resulting in excessive noise that overfit the source model.
Concentrating perturbation to dominant image regions that are model-agnostic is
crucial to improving adversarial efficacy. However, limiting perturbation to
local regions in the spatial domain proves inadequate in augmenting
transferability. To this end, we propose a transferable adversarial attack with
fine-grained perturbation optimization in the frequency domain, creating
centralized perturbation. We devise a systematic pipeline to dynamically
constrain perturbation optimization to dominant frequency coefficients. The
constraint is optimized in parallel at each iteration, ensuring the directional
alignment of perturbation optimization with model prediction. Our approach
allows us to centralize perturbation towards sample-specific important
frequency features, which are shared by DNNs, effectively mitigating source
model overfitting. Experiments demonstrate that by dynamically centralizing
perturbation on dominating frequency coefficients, crafted adversarial examples
exhibit stronger transferability, and allowing them to bypass various defenses.
