  Language models trained on billions of tokens have recently led to
unprecedented results on many NLP tasks. This success raises the question of
whether, in principle, a system can ever ``understand'' raw text without access
to some form of grounding. We formally investigate the abilities of ungrounded
systems to acquire meaning. Our analysis focuses on the role of ``assertions'':
textual contexts that provide indirect clues about the underlying semantics. We
study whether assertions enable a system to emulate representations preserving
semantic relations like equivalence. We find that assertions enable semantic
emulation of languages that satisfy a strong notion of semantic transparency.
However, for classes of languages where the same expression can take different
values in different contexts, we show that emulation can become uncomputable.
Finally, we discuss differences between our formal model and natural language,
exploring how our results generalize to a modal setting and other semantic
relations. Together, our results suggest that assertions in code or language do
not provide sufficient signal to fully emulate semantic representations. We
formalize ways in which ungrounded language models appear to be fundamentally
limited in their ability to ``understand''.
