  Despite the fact that beliefs are mental states that cannot be directly
observed, humans talk about each others' beliefs on a regular basis, often
using rich compositional language to describe what others think and know. What
explains this capacity to interpret the hidden epistemic content of other
minds? In this paper, we take a step towards an answer by grounding the
semantics of belief statements in a Bayesian theory-of-mind: By modeling how
humans jointly infer coherent sets of goals, beliefs, and plans that explain an
agent's actions, then evaluating statements about the agent's beliefs against
these inferences via epistemic logic, our framework provides a conceptual role
semantics for belief, explaining the gradedness and compositionality of human
belief attributions, as well as their intimate connection with goals and plans.
We evaluate this framework by studying how humans attribute goals and beliefs
while watching an agent solve a doors-and-keys gridworld puzzle that requires
instrumental reasoning about hidden objects. In contrast to pure logical
deduction, non-mentalizing baselines, and mentalizing that ignores the role of
instrumental plans, our model provides a much better fit to human goal and
belief attributions, demonstrating the importance of theory-of-mind for a
semantics of belief.
