  Large language models (LLMs) can solve complex multi-step problems, but
little is known about how these computations are implemented internally.
Motivated by this, we study how LLMs answer multi-hop queries such as "The
spouse of the performer of Imagine is". These queries require two information
extraction steps: a latent one for resolving the first hop ("the performer of
Imagine") into the bridge entity (John Lennon), and one for resolving the
second hop ("the spouse of John Lennon") into the target entity (Yoko Ono).
Understanding how the latent step is computed internally is key to
understanding the overall computation. By carefully analyzing the internal
computations of transformer-based LLMs, we discover that the bridge entity is
resolved in the early layers of the model. Then, only after this resolution,
the two-hop query is solved in the later layers. Because the second hop
commences in later layers, there could be cases where these layers no longer
encode the necessary knowledge for correctly predicting the answer. Motivated
by this, we propose a novel "back-patching" analysis method whereby a hidden
representation from a later layer is patched back to an earlier layer. We find
that in up to 57% of previously incorrect cases there exists a back-patch that
results in the correct generation of the answer, showing that the later layers
indeed sometimes lack the needed functionality. Overall our methods and
findings open further opportunities for understanding and improving latent
reasoning in transformer-based LLMs.
