  Background: The increasing deployment of Conversational Artificial
Intelligence (CAI) in mental health interventions necessitates an evaluation of
their efficacy in rectifying cognitive biases and recognizing affect in
human-AI interactions. These biases, including theory of mind and autonomy
biases, can exacerbate mental health conditions such as depression and anxiety.
  Objective: This study aimed to assess the effectiveness of therapeutic
chatbots (Wysa, Youper) versus general-purpose language models (GPT-3.5, GPT-4,
Gemini Pro) in identifying and rectifying cognitive biases and recognizing
affect in user interactions.
  Methods: The study employed virtual case scenarios simulating typical
user-bot interactions. Cognitive biases assessed included theory of mind biases
(anthropomorphism, overtrust, attribution) and autonomy biases (illusion of
control, fundamental attribution error, just-world hypothesis). Responses were
evaluated on accuracy, therapeutic quality, and adherence to Cognitive
Behavioral Therapy (CBT) principles, using an ordinal scale. The evaluation
involved double review by cognitive scientists and a clinical psychologist.
  Results: The study revealed that general-purpose chatbots outperformed
therapeutic chatbots in rectifying cognitive biases, particularly in overtrust
bias, fundamental attribution error, and just-world hypothesis. GPT-4 achieved
the highest scores across all biases, while therapeutic bots like Wysa scored
the lowest. Affect recognition showed similar trends, with general-purpose bots
outperforming therapeutic bots in four out of six biases. However, the results
highlight the need for further refinement of therapeutic chatbots to enhance
their efficacy and ensure safe, effective use in digital mental health
interventions. Future research should focus on improving affective response and
addressing ethical considerations in AI-based therapy.
