  Mathematical reasoning in large language models (LMs) has garnered
significant attention in recent work, but there is a limited understanding of
how these models process and store information related to arithmetic tasks
within their architecture. In order to improve our understanding of this aspect
of language models, we present a mechanistic interpretation of
Transformer-based LMs on arithmetic questions using a causal mediation analysis
framework. By intervening on the activations of specific model components and
measuring the resulting changes in predicted probabilities, we identify the
subset of parameters responsible for specific predictions. This provides
insights into how information related to arithmetic is processed by LMs. Our
experimental results indicate that LMs process the input by transmitting the
information relevant to the query from mid-sequence early layers to the final
token using the attention mechanism. Then, this information is processed by a
set of MLP modules, which generate result-related information that is
incorporated into the residual stream. To assess the specificity of the
observed activation dynamics, we compare the effects of different model
components on arithmetic queries with other tasks, including number retrieval
from prompts and factual knowledge questions.
