  Although great progress has been made on adversarial attacks for deep neural
networks (DNNs), their transferability is still unsatisfactory, especially for
targeted attacks. There are two problems behind that have been long overlooked:
1) the conventional setting of $T$ iterations with the step size of
$\epsilon/T$ to comply with the $\epsilon$-constraint. In this case, most of
the pixels are allowed to add very small noise, much less than $\epsilon$; and
2) usually manipulating pixel-wise noise. However, features of a pixel
extracted by DNNs are influenced by its surrounding regions, and different DNNs
generally focus on different discriminative regions in recognition. To tackle
these issues, our previous work proposes a patch-wise iterative method (PIM)
aimed at crafting adversarial examples with high transferability. Specifically,
we introduce an amplification factor to the step size in each iteration, and
one pixel's overall gradient overflowing the $\epsilon$-constraint is properly
assigned to its surrounding regions by a project kernel. But targeted attacks
aim to push the adversarial examples into the territory of a specific class,
and the amplification factor may lead to underfitting. Thus, we introduce the
temperature and propose a patch-wise++ iterative method (PIM++) to further
improve transferability without significantly sacrificing the performance of
the white-box attack. Our method can be generally integrated to any
gradient-based attack methods. Compared with the current state-of-the-art
attack methods, we significantly improve the success rate by 33.1\% for defense
models and 31.4\% for normally trained models on average.
