  Recent text-to-image diffusion models have shown surprising performance in
generating high-quality images. However, concerns have arisen regarding the
unauthorized data usage during the training or fine-tuning process. One example
is when a model trainer collects a set of images created by a particular artist
and attempts to train a model capable of generating similar images without
obtaining permission and giving credit to the artist. To address this issue, we
propose a method for detecting such unauthorized data usage by planting the
injected memorization into the text-to-image diffusion models trained on the
protected dataset. Specifically, we modify the protected images by adding
unique contents on these images using stealthy image warping functions that are
nearly imperceptible to humans but can be captured and memorized by diffusion
models. By analyzing whether the model has memorized the injected content
(i.e., whether the generated images are processed by the injected
post-processing function), we can detect models that had illegally utilized the
unauthorized data. Experiments on Stable Diffusion and VQ Diffusion with
different model training or fine-tuning methods (i.e, LoRA, DreamBooth, and
standard training) demonstrate the effectiveness of our proposed method in
detecting unauthorized data usages. Code:
https://github.com/ZhentingWang/DIAGNOSIS.
