  Large Language Models (LLMs) have demonstrated remarkable performance across
various information-seeking and reasoning tasks. These computational systems
drive state-of-the-art dialogue systems, such as ChatGPT and Bard. They also
carry substantial promise in meeting the growing demands of mental health care,
albeit relatively unexplored. As such, this study sought to examine LLMs'
capability to generate empathetic responses in conversations that emulate those
in a mental health counselling setting. We selected five LLMs: version 3.5 and
version 4 of the Generative Pre-training (GPT), Vicuna FastChat-T5, Pathways
Language Model (PaLM) version 2, and Falcon-7B-Instruct. Based on a simple
instructional prompt, these models responded to utterances derived from the
EmpatheticDialogues (ED) dataset. Using three empathy-related metrics, we
compared their responses to those from traditional response generation dialogue
systems, which were fine-tuned on the ED dataset, along with human-generated
responses. Notably, we discovered that responses from the LLMs were remarkably
more empathetic in most scenarios. We position our findings in light of
catapulting advancements in creating empathetic conversational systems.
