  Addition is perhaps one of the simplest arithmetic tasks one can think of and
is usually performed using the carrying over algorithm. This algorithm consists
of two tasks: adding digits in the same position and carrying over a one
whenever necessary. We study how transformer models implement this algorithm
and how the two aforementioned tasks are allocated to different parts of the
network. We first focus on two-layer encoder-only models and show that the
carrying over algorithm is implemented in a modular fashion. The first layer is
mostly responsible for adding digits in the same position. The second layer
first decides, in the attention, which positions need a carried one or not, and
then performs the carrying of the one in the final MLP. We provide a simple way
of precisely identifying which neurons are responsible for that task. This
implementation of the carrying over algorithm occurs across a range of
hyperparameters for two as well as three-layer models. For small decoder-only
models, we observe the same implementation and provide suggestive evidence for
its existence in three 7B large language models.
