  The advances of Large Language Models (LLMs) are expanding their utility in
both academic research and practical applications. Recent social science
research has explored the use of these ``black-box'' LLM agents for simulating
complex social systems and potentially substituting human subjects in
experiments. Our study delves into this emerging domain, investigating the
extent to which LLMs exhibit key social interaction principles, such as social
learning, social preference, and cooperative behavior (indirect reciprocity),
in their interactions with humans and other agents. We develop a framework for
our study, wherein classical laboratory experiments involving human subjects
are adapted to use LLM agents. This approach involves step-by-step reasoning
that mirrors human cognitive processes and zero-shot learning to assess the
innate preferences of LLMs. Our analysis of LLM agents' behavior includes both
the primary effects and an in-depth examination of the underlying mechanisms.
Focusing on GPT-4, our analyses suggest that LLM agents appear to exhibit a
range of human-like social behaviors such as distributional and reciprocity
preferences, responsiveness to group identity cues, engagement in indirect
reciprocity, and social learning capabilities. However, our analysis also
reveals notable differences: LLMs demonstrate a pronounced fairness preference,
weaker positive reciprocity, and a more calculating approach in social learning
compared to humans. These insights indicate that while LLMs hold great promise
for applications in social science research, such as in laboratory experiments
and agent-based modeling, the subtle behavioral differences between LLM agents
and humans warrant further investigation. Careful examination and development
of protocols in evaluating the social behaviors of LLMs are necessary before
directly applying these models to emulate human behavior.
