  The vulnerability of deep neural networks to small and even imperceptible
perturbations has become a central topic in deep learning research. Although
several sophisticated defense mechanisms have been introduced, most were later
shown to be ineffective. However, a reliable evaluation of model robustness is
mandatory for deployment in safety-critical scenarios. To overcome this problem
we propose a simple yet effective modification to the gradient calculation of
state-of-the-art first-order adversarial attacks. Normally, the gradient update
of an attack is directly calculated for the given data point. This approach is
sensitive to noise and small local optima of the loss function. Inspired by
gradient sampling techniques from non-convex optimization, we propose
Dynamically Sampled Nonlocal Gradient Descent (DSNGD). DSNGD calculates the
gradient direction of the adversarial attack as the weighted average over past
gradients of the optimization history. Moreover, distribution hyperparameters
that define the sampling operation are automatically learned during the
optimization scheme. We empirically show that by incorporating this nonlocal
gradient information, we are able to give a more accurate estimation of the
global descent direction on noisy and non-convex loss surfaces. In addition, we
show that DSNGD-based attacks are on average 35% faster while achieving 0.9% to
27.1% higher success rates compared to their gradient descent-based
counterparts.
