  Deep Neural Networks (DNNs) have been widely applied in many autonomous
systems such as autonomous driving. Recently, DNN testing has been intensively
studied to automatically generate adversarial examples, which inject
small-magnitude perturbations into inputs to test DNNs under extreme
situations. While existing testing techniques prove to be effective, they
mostly focus on generating digital adversarial perturbations (particularly for
autonomous driving), e.g., changing image pixels, which may never happen in
physical world. There is a critical missing piece in the literature on
autonomous driving testing: understanding and exploiting both digital and
physical adversarial perturbation generation for impacting steering decisions.
In this paper, we present DeepBillboard, a systematic physical-world testing
approach targeting at a common and practical driving scenario: drive-by
billboards. DeepBillboard is capable of generating a robust and resilient
printable adversarial billboard, which works under dynamic changing driving
conditions including viewing angle, distance, and lighting. The objective is to
maximize the possibility, degree, and duration of the steering-angle errors of
an autonomous vehicle driving by the generated adversarial billboard. We have
extensively evaluated the efficacy and robustness of DeepBillboard through
conducting both digital and physical-world experiments. Results show that
DeepBillboard is effective for various steering models and scenes. Furthermore,
DeepBillboard is sufficiently robust and resilient for generating
physical-world adversarial billboard tests for real-world driving under various
weather conditions. To the best of our knowledge, this is the first study
demonstrating the possibility of generating realistic and continuous
physical-world tests for practical autonomous driving systems.
