  Many adversarial attack methods achieve satisfactory attack success rates
under the white-box setting, but they usually show poor transferability when
attacking other DNN models. Momentum-based attack is one effective method to
improve transferability. It integrates the momentum term into the iterative
process, which can stabilize the update directions by adding the gradients'
temporal correlation for each pixel. We argue that only this temporal momentum
is not enough, the gradients from the spatial domain within an image, i.e.
gradients from the context pixels centered on the target pixel are also
important to the stabilization. For that, we propose a novel method named
Spatial Momentum Iterative FGSM attack (SMI-FGSM), which introduces the
mechanism of momentum accumulation from temporal domain to spatial domain by
considering the context information from different regions within the image.
SMI-FGSM is then integrated with temporal momentum to simultaneously stabilize
the gradients' update direction from both the temporal and spatial domains.
Extensive experiments show that our method indeed further enhances adversarial
transferability. It achieves the best transferability success rate for multiple
mainstream undefended and defended models, which outperforms the
state-of-the-art attack methods by a large margin of 10\% on average.
