  Modern natural language models such as the GPT-2/GPT-3 contain tremendous
amounts of information about human belief in a consistently testable form. If
these models could be shown to accurately reflect the underlying beliefs of the
human beings that produced the data used to train these models, then such
models become a powerful sociological tool in ways that are distinct from
traditional methods, such as interviews and surveys. In this study, We train a
version of the GPT-2 on a corpora of historical chess games, and then "launch"
clusters of synthetic agents into the model, using text strings to create
context and orientation. We compare the trajectories contained in the text
generated by the agents/model and compare that to the known ground truth of the
chess board, move legality, and historical patterns of play. We find that the
percentages of moves by piece using the model are substantially similar from
human patterns. We further find that the model creates an accurate latent
representation of the chessboard, and that it is possible to plot trajectories
of legal moves across the board using this knowledge.
