  In order for AI systems to communicate effectively with people, they must
understand how we make decisions. However, people's decisions are not always
rational, so the implicit internal models of human decision-making in Large
Language Models (LLMs) must account for this. Previous empirical evidence seems
to suggest that these implicit models are accurate -- LLMs offer believable
proxies of human behavior, acting how we expect humans would in everyday
interactions. However, by comparing LLM behavior and predictions to a large
dataset of human decisions, we find that this is actually not the case: when
both simulating and predicting people's choices, a suite of cutting-edge LLMs
(GPT-4o & 4-Turbo, Llama-3-8B & 70B, Claude 3 Opus) assume that people are more
rational than we really are. Specifically, these models deviate from human
behavior and align more closely with a classic model of rational choice --
expected value theory. Interestingly, people also tend to assume that other
people are rational when interpreting their behavior. As a consequence, when we
compare the inferences that LLMs and people draw from the decisions of others
using another psychological dataset, we find that these inferences are highly
correlated. Thus, the implicit decision-making models of LLMs appear to be
aligned with the human expectation that other people will act rationally,
rather than with how people actually act.
