  Adversarial testing methods based on Projected Gradient Descent (PGD) are
widely used for searching norm-bounded perturbations that cause the inputs of
neural networks to be misclassified. This paper takes a deeper look at these
methods and explains the effect of different hyperparameters (i.e., optimizer,
step size and surrogate loss). We introduce the concept of MultiTargeted
testing, which makes clever use of alternative surrogate losses, and explain
when and how MultiTargeted is guaranteed to find optimal perturbations.
Finally, we demonstrate that MultiTargeted outperforms more sophisticated
methods and often requires less iterative steps than other variants of PGD
found in the literature. Notably, MultiTargeted ranks first on MadryLab's
white-box MNIST and CIFAR-10 leaderboards, reducing the accuracy of their MNIST
model to 88.36% (with $\ell_\infty$ perturbations of $\epsilon = 0.3$) and the
accuracy of their CIFAR-10 model to 44.03% (at $\epsilon = 8/255$).
MultiTargeted also ranks first on the TRADES leaderboard reducing the accuracy
of their CIFAR-10 model to 53.07% (with $\ell_\infty$ perturbations of
$\epsilon = 0.031$).
