  The recent advances in diffusion models (DMs) have revolutionized the
generation of realistic and complex images. However, these models also
introduce potential safety hazards, such as producing harmful content and
infringing data copyrights. Despite the development of safety-driven unlearning
techniques to counteract these challenges, doubts about their efficacy persist.
To tackle this issue, we introduce an evaluation framework that leverages
adversarial prompts to discern the trustworthiness of these safety-driven DMs
after they have undergone the process of unlearning harmful concepts.
Specifically, we investigated the adversarial robustness of DMs, assessed by
adversarial prompts, when eliminating unwanted concepts, styles, and objects.
We develop an effective and efficient adversarial prompt generation approach
for DMs, termed UnlearnDiffAtk. This method capitalizes on the intrinsic
classification abilities of DMs to simplify the creation of adversarial
prompts, thereby eliminating the need for auxiliary classification or diffusion
models. Through extensive benchmarking, we evaluate the robustness of
widely-used safety-driven unlearned DMs (i.e., DMs after unlearning undesirable
concepts, styles, or objects) across a variety of tasks. Our results
demonstrate the effectiveness and efficiency merits of UnlearnDiffAtk over the
state-of-the-art adversarial prompt generation method and reveal the lack of
robustness of current safetydriven unlearning techniques when applied to DMs.
Codes are available at https://github.com/OPTML-Group/Diffusion-MU-Attack.
WARNING: There exist AI generations that may be offensive in nature.
