  Recent work has shown exciting promise in updating large language models with
new memories, so as to replace obsolete information or add specialized
knowledge. However, this line of work is predominantly limited to updating
single associations. We develop MEMIT, a method for directly updating a
language model with many memories, demonstrating experimentally that it can
scale up to thousands of associations for GPT-J (6B) and GPT-NeoX (20B),
exceeding prior work by orders of magnitude. Our code and data are at
https://memit.baulab.info.
