  Language models learn a great quantity of factual information during
pretraining, and recent work localizes this information to specific model
weights like mid-layer MLP weights. In this paper, we find that we can change
how a fact is stored in a model by editing weights that are in a different
location than where existing methods suggest that the fact is stored. This is
surprising because we would expect that localizing facts to specific model
parameters would tell us where to manipulate knowledge in models, and this
assumption has motivated past work on model editing methods. Specifically, we
show that localization conclusions from representation denoising (also known as
Causal Tracing) do not provide any insight into which model MLP layer would be
best to edit in order to override an existing stored fact with a new one. This
finding raises questions about how past work relies on Causal Tracing to select
which model layers to edit. Next, we consider several variants of the editing
problem, including erasing and amplifying facts. For one of our editing
problems, editing performance does relate to localization results from
representation denoising, but we find that which layer we edit is a far better
predictor of performance. Our results suggest, counterintuitively, that better
mechanistic understanding of how pretrained language models work may not always
translate to insights about how to best change their behavior. Our code is
available at https://github.com/google/belief-localization
