  Humans can attribute beliefs to others. However, it is unknown to what extent
this ability results from an innate biological endowment or from experience
accrued through child development, particularly exposure to language describing
others' mental states. We test the viability of the language exposure
hypothesis by assessing whether models exposed to large quantities of human
language display sensitivity to the implied knowledge states of characters in
written passages. In pre-registered analyses, we present a linguistic version
of the False Belief Task to both human participants and a Large Language Model,
GPT-3. Both are sensitive to others' beliefs, but while the language model
significantly exceeds chance behavior, it does not perform as well as the
humans, nor does it explain the full extent of their behavior -- despite being
exposed to more language than a human would in a lifetime. This suggests that
while statistical learning from language exposure may in part explain how
humans develop the ability to reason about the mental states of others, other
mechanisms are also responsible.
