  A significant amount of work has been done on adversarial attacks that inject
imperceptible noise to images to deteriorate the image classification
performance of deep models. However, most of the existing studies consider
attacks in the digital (pixel) domain where an image acquired by an image
sensor with sampling and quantization has been recorded. This paper, for the
first time, introduces an optical adversarial attack, which physically alters
the light field information arriving at the image sensor so that the
classification model yields misclassification. More specifically, we modulate
the phase of the light in the Fourier domain using a spatial light modulator
placed in the photographic system. The operative parameters of the modulator
are obtained by gradient-based optimization to maximize cross-entropy and
minimize distortions. We present experiments based on both simulation and a
real hardware optical system, from which the feasibility of the proposed
optical attack is demonstrated. It is also verified that the proposed attack is
completely different from common optical-domain distortions such as spherical
aberration, defocus, and astigmatism in terms of both perturbation patterns and
classification results.
