  Do language models have beliefs about the world? Dennett (1995) famously
argues that even thermostats have beliefs, on the view that a belief is simply
an informational state decoupled from any motivational state. In this paper, we
discuss approaches to detecting when models have beliefs about the world, and
we improve on methods for updating model beliefs to be more truthful, with a
focus on methods based on learned optimizers or hypernetworks. Our main
contributions include: (1) new metrics for evaluating belief-updating methods
that focus on the logical consistency of beliefs, (2) a training objective for
Sequential, Local, and Generalizing model updates (SLAG) that improves the
performance of learned optimizers, and (3) the introduction of the belief
graph, which is a new form of interface with language models that shows the
interdependencies between model beliefs. Our experiments suggest that models
possess belief-like qualities to only a limited extent, but update methods can
both fix incorrect model beliefs and greatly improve their consistency.
Although off-the-shelf optimizers are surprisingly strong belief-updating
baselines, our learned optimizers can outperform them in more difficult
settings than have been considered in past work. Code is available at
https://github.com/peterbhase/SLAG-Belief-Updating
