  Recent advances in large text-conditional image generative models such as
Stable Diffusion, Midjourney, and DALL-E 3 have revolutionized the field of
image generation, allowing users to produce high-quality, realistic images from
textual prompts. While these developments have enhanced artistic creation and
visual communication, they also present an underexplored attack opportunity:
the possibility of inducing biases by an adversary into the generated images
for malicious intentions, e.g., to influence society and spread propaganda. In
this paper, we demonstrate the possibility of such a bias injection threat by
an adversary who backdoors such models with a small number of malicious data
samples; the implemented backdoor is activated when special triggers exist in
the input prompt of the backdoored models. On the other hand, the model's
utility is preserved in the absence of the triggers, making the attack highly
undetectable. We present a novel framework that enables efficient generation of
poisoning samples with composite (multi-word) triggers for such an attack. Our
extensive experiments using over 1 million generated images and against
hundreds of fine-tuned models demonstrate the feasibility of the presented
backdoor attack. We illustrate how these biases can bypass conventional
detection mechanisms, highlighting the challenges in proving the existence of
biases within operational constraints. Our cost analysis confirms the low
financial barrier to executing such attacks, underscoring the need for robust
defensive strategies against such vulnerabilities in text-to-image generation
models.
