  Physical world adversarial attack is a highly practical and threatening
attack, which fools real world deep learning systems by generating conspicuous
and maliciously crafted real world artifacts. In physical world attacks,
evaluating naturalness is highly emphasized since human can easily detect and
remove unnatural attacks. However, current studies evaluate naturalness in a
case-by-case fashion, which suffers from errors, bias and inconsistencies. In
this paper, we take the first step to benchmark and assess visual naturalness
of physical world attacks, taking autonomous driving scenario as the first
attempt. First, to benchmark attack naturalness, we contribute the first
Physical Attack Naturalness (PAN) dataset with human rating and gaze. PAN
verifies several insights for the first time: naturalness is (disparately)
affected by contextual features (i.e., environmental and semantic variations)
and correlates with behavioral feature (i.e., gaze signal). Second, to
automatically assess attack naturalness that aligns with human ratings, we
further introduce Dual Prior Alignment (DPA) network, which aims to embed human
knowledge into model reasoning process. Specifically, DPA imitates human
reasoning in naturalness assessment by rating prior alignment and mimics human
gaze behavior by attentive prior alignment. We hope our work fosters researches
to improve and automatically assess naturalness of physical world attacks. Our
code and dataset can be found at https://github.com/zhangsn-19/PAN.
