  Light-based adversarial attacks use spatial augmented reality (SAR)
techniques to fool image classifiers by altering the physical light condition
with a controllable light source, e.g., a projector. Compared with physical
attacks that place hand-crafted adversarial objects, projector-based ones
obviate modifying the physical entities, and can be performed transiently and
dynamically by altering the projection pattern. However, subtle light
perturbations are insufficient to fool image classifiers, due to the complex
environment and project-and-capture process. Thus, existing approaches focus on
projecting clearly perceptible adversarial patterns, while the more interesting
yet challenging goal, stealthy projector-based attack, remains open. In this
paper, for the first time, we formulate this problem as an end-to-end
differentiable process and propose a Stealthy Projector-based Adversarial
Attack (SPAA) solution. In SPAA, we approximate the real Project-and-Capture
process using a deep neural network named PCNet, then we include PCNet in the
optimization of projector-based attacks such that the generated adversarial
projection is physically plausible. Finally, to generate both robust and
stealthy adversarial projections, we propose an algorithm that uses minimum
perturbation and adversarial confidence thresholds to alternate between the
adversarial loss and stealthiness loss optimization. Our experimental
evaluations show that SPAA clearly outperforms other methods by achieving
higher attack success rates and meanwhile being stealthier, for both targeted
and untargeted attacks.
