  We explore the topology of representation manifolds arising in autoregressive
neural language models trained on raw text data. In order to study their
properties, we introduce tools from computational algebraic topology, which we
use as a basis for a measure of topological complexity, that we call
perforation.
  Using this measure, we study the evolution of topological structure in GPT
based large language models across depth and time during training. We then
compare these to gated recurrent models, and show that the latter exhibit more
topological complexity, with a distinct pattern of changes common to all
natural languages but absent from synthetically generated data. The paper
presents a detailed analysis of the representation manifolds derived by these
models based on studying the shapes of vector clouds induced by them as they
are conditioned on sentences from corpora of natural language text.
  The methods developed in this paper are novel in the field and based on
mathematical apparatus that might be unfamiliar to the target audience. To help
with that we introduce the minimum necessary theory, and provide additional
visualizations in the appendices.
  The main contribution of the paper is a striking observation about the
topological structure of the transformer as compared to LSTM based neural
architectures. It suggests that further research into mathematical properties
of these neural networks is necessary to understand the operation of large
transformer language models. We hope this work inspires further explorations in
this direction within the NLP community.
