  State of the art language models return a natural language text continuation
from any piece of input text. This ability to generate coherent text extensions
implies significant sophistication, including a knowledge of grammar and
semantics. In this paper, we propose a mathematical framework for passing from
probability distributions on extensions of given texts, such as the ones
learned by today's large language models, to an enriched category containing
semantic information. Roughly speaking, we model probability distributions on
texts as a category enriched over the unit interval. Objects of this category
are expressions in language, and hom objects are conditional probabilities that
one expression is an extension of another. This category is syntactical -- it
describes what goes with what. Then, via the Yoneda embedding, we pass to the
enriched category of unit interval-valued copresheaves on this syntactical
category. This category of enriched copresheaves is semantic -- it is where we
find meaning, logical operations such as entailment, and the building blocks
for more elaborate semantic concepts.
