  Identifying layers within text-to-image models which control visual
attributes can facilitate efficient model editing through closed-form updates.
Recent work, leveraging causal tracing show that early Stable-Diffusion
variants confine knowledge primarily to the first layer of the CLIP
text-encoder, while it diffuses throughout the UNet.Extending this framework,
we observe that for recent models (e.g., SD-XL, DeepFloyd), causal tracing
fails in pinpointing localized knowledge, highlighting challenges in model
editing. To address this issue, we introduce the concept of Mechanistic
Localization in text-to-image models, where knowledge about various visual
attributes (e.g., "style", "objects", "facts") can be mechanistically localized
to a small fraction of layers in the UNet, thus facilitating efficient model
editing. We localize knowledge using our method LocoGen which measures the
direct effect of intermediate layers to output generation by performing
interventions in the cross-attention layers of the UNet. We then employ
LocoEdit, a fast closed-form editing method across popular open-source
text-to-image models (including the latest SD-XL)and explore the possibilities
of neuron-level model editing. Using Mechanistic Localization, our work offers
a better view of successes and failures in localization-based text-to-image
model editing. Code will be available at
https://github.com/samyadeepbasu/LocoGen.
