  Concept erasure in text-to-image diffusion models aims to disable pre-trained
diffusion models from generating images related to a target concept. To perform
reliable concept erasure, the properties of robustness and locality are
desirable. The former refrains the model from producing images associated with
the target concept for any paraphrased or learned prompts, while the latter
preserves its ability in generating images with non-target concepts. In this
paper, we propose Reliable Concept Erasing via Lightweight Erasers (Receler).
It learns a lightweight Eraser to perform concept erasing while satisfying the
above desirable properties through the proposed concept-localized
regularization and adversarial prompt learning scheme. Experiments with various
concepts verify the superiority of Receler over previous methods.
