  Face recognition (FR) models can be easily fooled by adversarial examples,
which are crafted by adding imperceptible perturbations on benign face images.
The existence of adversarial face examples poses a great threat to the security
of society. In order to build a more sustainable digital nation, in this paper,
we improve the transferability of adversarial face examples to expose more
blind spots of existing FR models. Though generating hard samples has shown its
effectiveness in improving the generalization of models in training tasks, the
effectiveness of utilizing this idea to improve the transferability of
adversarial face examples remains unexplored. To this end, based on the
property of hard samples and the symmetry between training tasks and
adversarial attack tasks, we propose the concept of hard models, which have
similar effects as hard samples for adversarial attack tasks. Utilizing the
concept of hard models, we propose a novel attack method called Beneficial
Perturbation Feature Augmentation Attack (BPFA), which reduces the overfitting
of adversarial examples to surrogate FR models by constantly generating new
hard models to craft the adversarial examples. Specifically, in the
backpropagation, BPFA records the gradients on pre-selected feature maps and
uses the gradient on the input image to craft the adversarial example. In the
next forward propagation, BPFA leverages the recorded gradients to add
beneficial perturbations on their corresponding feature maps to increase the
loss. Extensive experiments demonstrate that BPFA can significantly boost the
transferability of adversarial attacks on FR.
