  Deep neural networks (DNNs) are playing key roles in various artificial
intelligence applications such as image classification and object recognition.
However, a growing number of studies have shown that there exist adversarial
examples in DNNs, which are almost imperceptibly different from original
samples, but can greatly change the network output. Existing white-box attack
algorithms can generate powerful adversarial examples. Nevertheless, most of
the algorithms concentrate on how to iteratively make the best use of gradients
to improve adversarial performance. In contrast, in this paper, we focus on the
properties of the widely-used ReLU activation function, and discover that there
exist two phenomena (i.e., wrong blocking and over transmission) misleading the
calculation of gradients in ReLU during the backpropagation. Both issues
enlarge the difference between the predicted changes of the loss function from
gradient and corresponding actual changes, and mislead the gradients which
results in larger perturbations. Therefore, we propose a universal adversarial
example generation method, called ADV-ReLU, to enhance the performance of
gradient based white-box attack algorithms. During the backpropagation of the
network, our approach calculates the gradient of the loss function versus
network input, maps the values to scores, and selects a part of them to update
the misleading gradients. Comprehensive experimental results on \emph{ImageNet}
demonstrate that our ADV-ReLU can be easily integrated into many
state-of-the-art gradient-based white-box attack algorithms, as well as
transferred to black-box attack attackers, to further decrease perturbations in
the ${\ell _2}$-norm.
