  This paper investigates the empathetic responding capabilities of ChatGPT,
particularly its latest iteration, GPT-4, in comparison to human-generated
responses to a wide range of emotional scenarios, both positive and negative.
We employ a rigorous evaluation methodology, involving a between-groups study
with 600 participants, to evaluate the level of empathy in responses generated
by humans and ChatGPT. ChatGPT is prompted in two distinct ways: a standard
approach and one explicitly detailing empathy's cognitive, affective, and
compassionate counterparts. Our findings indicate that the average empathy
rating of responses generated by ChatGPT exceeds those crafted by humans by
approximately 10%. Additionally, instructing ChatGPT to incorporate a clear
understanding of empathy in its responses makes the responses align
approximately 5 times more closely with the expectations of individuals
possessing a high degree of empathy, compared to human responses. The proposed
evaluation framework serves as a scalable and adaptable framework to assess the
empathetic capabilities of newer and updated versions of large language models,
eliminating the need to replicate the current study's results in future
research.
