  Denoising probabilistic diffusion models have shown breakthrough performance
to generate more photo-realistic images or human-level illustrations than the
prior models such as GANs. This high image-generation capability has stimulated
the creation of many downstream applications in various areas. However, we find
that this technology is actually a double-edged sword: We identify a new type
of attack, called the Natural Denoising Diffusion (NDD) attack based on the
finding that state-of-the-art deep neural network (DNN) models still hold their
prediction even if we intentionally remove their robust features, which are
essential to the human visual system (HVS), through text prompts. The NDD
attack shows a significantly high capability to generate low-cost,
model-agnostic, and transferable adversarial attacks by exploiting the natural
attack capability in diffusion models. To systematically evaluate the risk of
the NDD attack, we perform a large-scale empirical study with our newly created
dataset, the Natural Denoising Diffusion Attack (NDDA) dataset. We evaluate the
natural attack capability by answering 6 research questions. Through a user
study, we find that it can achieve an 88% detection rate while being stealthy
to 93% of human subjects; we also find that the non-robust features embedded by
diffusion models contribute to the natural attack capability. To confirm the
model-agnostic and transferable attack capability, we perform the NDD attack
against the Tesla Model 3 and find that 73% of the physically printed attacks
can be detected as stop signs. Our hope is that the study and dataset can help
our community be aware of the risks in diffusion models and facilitate further
research toward robust DNN models.
