  Advances in artificial intelligence (AI) are enabling systems that augment
and collaborate with humans to perform simple, mechanistic tasks like
scheduling meetings and grammar-checking text. However, such Human-AI
collaboration poses challenges for more complex, creative tasks, such as
carrying out empathic conversations, due to difficulties of AI systems in
understanding complex human emotions and the open-ended nature of these tasks.
Here, we focus on peer-to-peer mental health support, a setting in which
empathy is critical for success, and examine how AI can collaborate with humans
to facilitate peer empathy during textual, online supportive conversations. We
develop Hailey, an AI-in-the-loop agent that provides just-in-time feedback to
help participants who provide support (peer supporters) respond more
empathically to those seeking help (support seekers). We evaluate Hailey in a
non-clinical randomized controlled trial with real-world peer supporters on
TalkLife (N=300), a large online peer-to-peer support platform. We show that
our Human-AI collaboration approach leads to a 19.60% increase in
conversational empathy between peers overall. Furthermore, we find a larger
38.88% increase in empathy within the subsample of peer supporters who
self-identify as experiencing difficulty providing support. We systematically
analyze the Human-AI collaboration patterns and find that peer supporters are
able to use the AI feedback both directly and indirectly without becoming
overly reliant on AI while reporting improved self-efficacy post-feedback. Our
findings demonstrate the potential of feedback-driven, AI-in-the-loop writing
systems to empower humans in open-ended, social, creative tasks such as
empathic conversations.
