  Face verification (FV) using deep neural network models has made tremendous
progress in recent years, surpassing human accuracy and seeing deployment in
various applications such as border control and smartphone unlocking. However,
FV systems are vulnerable to Adversarial Attacks, which manipulate input images
to deceive these systems in ways usually unnoticeable to humans. This paper
provides an in-depth study of attacks on FV systems. We introduce the
DodgePersonation Attack that formulates the creation of face images that
impersonate a set of given identities while avoiding being identified as any of
the identities in a separate, disjoint set. A taxonomy is proposed to provide a
unified view of different types of Adversarial Attacks against FV systems,
including Dodging Attacks, Impersonation Attacks, and Master Face Attacks.
Finally, we propose the ''One Face to Rule Them All'' Attack which implements
the DodgePersonation Attack with state-of-the-art performance on a well-known
scenario (Master Face Attack) and which can also be used for the new scenarios
introduced in this paper. While the state-of-the-art Master Face Attack can
produce a set of 9 images to cover 43.82% of the identities in their test
database, with 9 images our attack can cover 57.27% to 58.5% of these
identifies while giving the attacker the choice of the identity to use to
create the impersonation. Moreover, the 9 generated attack images appear
identical to a casual observer.
