  The wide deployment of Face Recognition (FR) systems poses risks of privacy
leakage. One countermeasure to address this issue is adversarial attacks, which
deceive malicious FR searches but simultaneously interfere the normal identity
verification of trusted authorizers. In this paper, we propose the first Double
Privacy Guard (DPG) scheme based on traceable adversarial watermarking. DPG
employs a one-time watermark embedding to deceive unauthorized FR models and
allows authorizers to perform identity verification by extracting the
watermark. Specifically, we propose an information-guided adversarial attack
against FR models. The encoder embeds an identity-specific watermark into the
deep feature space of the carrier, guiding recognizable features of the image
to deviate from the source identity. We further adopt a collaborative
meta-optimization strategy compatible with sub-tasks, which regularizes the
joint optimization direction of the encoder and decoder. This strategy enhances
the representation of universal carrier features, mitigating multi-objective
optimization conflicts in watermarking. Experiments confirm that DPG achieves
significant attack success rates and traceability accuracy on state-of-the-art
FR models, exhibiting remarkable robustness that outperforms the existing
privacy protection methods using adversarial attacks and deep watermarking, or
simple combinations of the two. Our work potentially opens up new insights into
proactive protection for FR privacy.
