  The increasingly pervasive facial recognition (FR) systems raise serious
concerns about personal privacy, especially for billions of users who have
publicly shared their photos on social media. Several attempts have been made
to protect individuals from unauthorized FR systems utilizing adversarial
attacks to generate encrypted face images to protect users from being
identified by FR systems. However, existing methods suffer from poor visual
quality or low attack success rates, which limit their usability in practice.
In this paper, we propose Attribute Guided Encryption with Facial Texture
Masking (AGE-FTM) that performs a dual manifold adversarial attack on FR
systems to achieve both good visual quality and high black box attack success
rates. In particular, AGE-FTM utilizes a high fidelity generative adversarial
network (GAN) to generate natural on-manifold adversarial samples by modifying
facial attributes, and performs the facial texture masking attack to generate
imperceptible off-manifold adversarial samples. Extensive experiments on the
CelebA-HQ dataset demonstrate that our proposed method produces more
natural-looking encrypted images than state-of-the-art methods while achieving
competitive attack performance. We further evaluate the effectiveness of
AGE-FTM in the real world using a commercial FR API and validate its usefulness
in practice through an user study.
