  2D face recognition has been proven insecure for physical adversarial
attacks. However, few studies have investigated the possibility of attacking
real-world 3D face recognition systems. 3D-printed attacks recently proposed
cannot generate adversarial points in the air. In this paper, we attack 3D face
recognition systems through elaborate optical noises. We took structured light
3D scanners as our attack target. End-to-end attack algorithms are designed to
generate adversarial illumination for 3D faces through the inherent or an
additional projector to produce adversarial points at arbitrary positions.
Nevertheless, face reflectance is a complex procedure because the skin is
translucent. To involve this projection-and-capture procedure in optimization
loops, we model it by Lambertian rendering model and use SfSNet to estimate the
albedo. Moreover, to improve the resistance to distance and angle changes while
maintaining the perturbation unnoticeable, a 3D transform invariant loss and
two kinds of sensitivity maps are introduced. Experiments are conducted in both
simulated and physical worlds. We successfully attacked point-cloud-based and
depth-image-based 3D face recognition algorithms while needing fewer
perturbations than previous state-of-the-art physical-world 3D adversarial
attacks.
