  The premise of this article is that a basic understanding of the composition
and functioning of large language models is critically urgent. To that end, we
extract a representational map of OpenAI's GPT-2 with what we articulate as two
classes of deep learning code, that which pertains to the model and that which
underwrites applications built around the model. We then verify this map
through case studies of two popular GPT-2 applications: the text adventure
game, AI Dungeon, and the language art project, This Word Does Not Exist. Such
an exercise allows us to test the potential of Critical Code Studies when the
object of study is deep learning code and to demonstrate the validity of code
as an analytical focus for researchers in the subfields of Critical Artificial
Intelligence and Critical Machine Learning Studies. More broadly, however, our
work draws attention to the means by which ordinary users might interact with,
and even direct, the behavior of deep learning systems, and by extension works
toward demystifying some of the auratic mystery of "AI." What is at stake is
the possibility of achieving an informed sociotechnical consensus about the
responsible applications of large language models, as well as a more expansive
sense of their creative capabilities-indeed, understanding how and where
engagement occurs allows all of us to become more active participants in the
development of machine learning systems.
