  Theory of Mind (ToM), the capacity to comprehend the mental states of
distinct individuals, is essential for numerous practical applications. With
the development of large language models (LLMs), there is a heated debate about
whether they are able to perform ToM tasks. Previous studies have used
different tasks and prompts to test the ToM on LLMs and the results are
inconsistent: some studies asserted these models are capable of exhibiting ToM,
while others suggest the opposite. In this study, We present ToMChallenges, a
dataset for comprehensively evaluating the Theory of Mind based on the
Sally-Anne and Smarties tests with a diverse set of tasks. In addition, we also
propose an auto-grader to streamline the answer evaluation process. We tested
three models: davinci, turbo, and gpt-4. Our evaluation results and error
analyses show that LLMs have inconsistent behaviors across prompts and tasks.
Performing the ToM tasks robustly remains a challenge for the LLMs. In
addition, our paper wants to raise awareness in evaluating the ToM in LLMs and
we want to invite more discussion on how to design the prompts and tasks for
ToM tasks that can better assess the LLMs' ability.
