  Publicly deploying research chatbots is a nuanced topic involving necessary
risk-benefit analyses. While there have recently been frequent discussions on
whether it is responsible to deploy such models, there has been far less focus
on the interaction paradigms and design approaches that the resulting
interfaces should adopt, in order to achieve their goals more effectively. We
aim to pose, ground, and attempt to answer HCI questions involved in this
scope, by reporting on a mixed-methods user study conducted on a recent
research chatbot. We find that abstract anthropomorphic representation for the
agent has a significant effect on user's perception, that offering AI
explainability may have an impact on feedback rates, and that two (diegetic and
extradiegetic) levels of the chat experience should be intentionally designed.
We offer design recommendations and areas of further focus for the research
community.
