  Deep face recognition (FR) has achieved significantly high accuracy on
several challenging datasets and fosters successful real-world applications,
even showing high robustness to the illumination variation that is usually
regarded as a main threat to the FR system. However, in the real world,
illumination variation caused by diverse lighting conditions cannot be fully
covered by the limited face dataset. In this paper, we study the threat of
lighting against FR from a new angle, i.e., adversarial attack, and identify a
new task, i.e., adversarial relighting. Given a face image, adversarial
relighting aims to produce a naturally relighted counterpart while fooling the
state-of-the-art deep FR methods. To this end, we first propose the physical
modelbased adversarial relighting attack (ARA) denoted as albedoquotient-based
adversarial relighting attack (AQ-ARA). It generates natural adversarial light
under the physical lighting model and guidance of FR systems and synthesizes
adversarially relighted face images. Moreover, we propose the auto-predictive
adversarial relighting attack (AP-ARA) by training an adversarial relighting
network (ARNet) to automatically predict the adversarial light in a one-step
manner according to different input faces, allowing efficiency-sensitive
applications. More importantly, we propose to transfer the above digital
attacks to physical ARA (PhyARA) through a precise relighting device, making
the estimated adversarial lighting condition reproducible in the real world. We
validate our methods on three state-of-the-art deep FR methods, i.e., FaceNet,
ArcFace, and CosFace, on two public datasets. The extensive and insightful
results demonstrate our work can generate realistic adversarial relighted face
images fooling face recognition tasks easily, revealing the threat of specific
light directions and strengths.
