  As Large Language Models (LLMs) are deployed more widely, customization with
respect to vocabulary, style, and character becomes more important. In this
work, we introduce model arithmetic, a novel inference framework for composing
and biasing LLMs without the need for model (re)training or highly specific
datasets. In addition, the framework allows for more precise control of
generated text than direct prompting and prior controlled text generation (CTG)
techniques. Using model arithmetic, we can express prior CTG techniques as
simple formulas and naturally extend them to new and more effective
formulations. Further, we show that speculative sampling, a technique for
efficient LLM sampling, extends to our setting. This enables highly efficient
text generation with multiple composed models with only marginal overhead over
a single model. Our empirical evaluation demonstrates that model arithmetic
allows fine-grained control of generated text while outperforming
state-of-the-art on the task of toxicity reduction. We release an open source
easy-to-use implementation of our framework at
https://github.com/eth-sri/language-model-arithmetic.
