  Theory-of-mind (ToM), a human ability to infer the intentions and thoughts of
others, is an essential part of empathetic experiences. We provide here the
framework for using NLP models to measure ToM expressed in written texts. For
this purpose, we introduce ToM-Diary, a crowdsourced 18,238 diaries with 74,014
Korean sentences annotated with different ToM levels. Each diary was annotated
with ToM levels by trained psychology students and reviewed by selected
psychology experts. The annotators first divided the diaries based on whether
they mentioned other people: self-focused and other-focused. Examples of
self-focused sentences are "I am feeling good". The other-focused sentences
were further classified into different levels. These levels differ by whether
the writer 1) mentions the presence of others without inferring their mental
state(e.g., I saw a man walking down the street), 2) fails to take the
perspective of others (e.g., I don't understand why they refuse to wear masks),
or 3) successfully takes the perspective of others (It must have been hard for
them to continue working). We tested whether state-of-the-art transformer-based
models (e.g., BERT) could predict underlying ToM levels in sentences. We found
that BERT more successfully detected self-focused sentences than other-focused
ones. Sentences that successfully take the perspective of others (the highest
ToM level) were the most difficult to predict. Our study suggests a promising
direction for large-scale and computational approaches for identifying the
ability of authors to empathize and take the perspective of others. The dataset
is at [URL](https://github.com/humanfactorspsych/covid19-tom-empathy-diary)
