  In recent years, diffusion models have achieved remarkable success in the
realm of high-quality image generation, garnering increased attention. This
surge in interest is paralleled by a growing concern over the security threats
associated with diffusion models, largely attributed to their susceptibility to
malicious exploitation. Notably, recent research has brought to light the
vulnerability of diffusion models to backdoor attacks, enabling the generation
of specific target images through corresponding triggers. However, prevailing
backdoor attack methods rely on manually crafted trigger generation functions,
often manifesting as discernible patterns incorporated into input noise, thus
rendering them susceptible to human detection. In this paper, we present an
innovative and versatile optimization framework designed to acquire invisible
triggers, enhancing the stealthiness and resilience of inserted backdoors. Our
proposed framework is applicable to both unconditional and conditional
diffusion models, and notably, we are the pioneers in demonstrating the
backdooring of diffusion models within the context of text-guided image editing
and inpainting pipelines. Moreover, we also show that the backdoors in the
conditional generation can be directly applied to model watermarking for model
ownership verification, which further boosts the significance of the proposed
framework. Extensive experiments on various commonly used samplers and datasets
verify the efficacy and stealthiness of the proposed framework. Our code is
publicly available at
https://github.com/invisibleTriggerDiffusion/invisible_triggers_for_diffusion.
