  Previous researches on dialogue system assessment usually focus on the
quality evaluation (e.g. fluency, relevance, etc) of responses generated by the
chatbots, which are local and technical metrics. For a chatbot which responds
to millions of online users including minors, we argue that it should have a
healthy mental tendency in order to avoid the negative psychological impact on
them. In this paper, we establish several mental health assessment dimensions
for chatbots (depression, anxiety, alcohol addiction, empathy) and introduce
the questionnaire-based mental health assessment methods. We conduct
assessments on some well-known open-domain chatbots and find that there are
severe mental health issues for all these chatbots. We consider that it is due
to the neglect of the mental health risks during the dataset building and the
model training procedures. We expect to attract researchers' attention to the
serious mental health problems of chatbots and improve the chatbots' ability in
positive emotional interaction.
