  Theory of Mind (ToM) is a critical component of intelligence but its
assessment remains the subject of heated debates. Prior research applied human
ToM assessments to natural language processing models using either
human-created standardized tests or rule-based templates. However, these
methods primarily focus on simplistic reasoning and require further validation.
Here, we leverage dynamic epistemic logic to isolate a particular component of
ToM and to generate controlled problems. We also introduce new verbalization
techniques to express these problems in English natural language. Our findings
indicate that some language model scaling (from 70M to 6B and 350M to 174B)
does not consistently yield results better than random chance. While GPT-4
demonstrates superior epistemic reasoning capabilities, there is still room for
improvement. Our code and datasets are publicly available
(https://huggingface.co/datasets/sileod/mindgames ,
https://github.com/sileod/llm-theory-of-mind )
