  Deep neural networks are vulnerable to adversarial examples, which can fool
deep models by adding subtle perturbations. Although existing attacks have
achieved promising results, it still leaves a long way to go for generating
transferable adversarial examples under the black-box setting. To this end,
this paper proposes to improve the transferability of adversarial examples, and
applies dual-stage feature-level perturbations to an existing model to
implicitly create a set of diverse models. Then these models are fused by the
longitudinal ensemble during the iterations. The proposed method is termed
Dual-Stage Network Erosion (DSNE). We conduct comprehensive experiments both on
non-residual and residual networks, and obtain more transferable adversarial
examples with the computational cost similar to the state-of-the-art method. In
particular, for the residual networks, the transferability of the adversarial
examples can be significantly improved by biasing the residual block
information to the skip connections. Our work provides new insights into the
architectural vulnerability of neural networks and presents new challenges to
the robustness of neural networks.
