  Large Language Models (LLMs) have become valuable assets in mental health,
showing promise in both classification tasks and counseling applications. This
paper offers a perspective on using LLMs in mental health applications. It
discusses the instability of generative models for prediction and the potential
for generating hallucinatory outputs, underscoring the need for ongoing audits
and evaluations to maintain their reliability and dependability. The paper also
distinguishes between the often interchangeable terms ``explainability'' and
``interpretability'', advocating for developing inherently interpretable
methods instead of relying on potentially hallucinated self-explanations
generated by LLMs. Despite the advancements in LLMs, human counselors'
empathetic understanding, nuanced interpretation, and contextual awareness
remain irreplaceable in the sensitive and complex realm of mental health
counseling. The use of LLMs should be approached with a judicious and
considerate mindset, viewing them as tools that complement human expertise
rather than seeking to replace it.
