  Social robotics researchers are increasingly interested in multi-party
trained conversational agents. With a growing demand for real-world
evaluations, our study presents Large Language Models (LLMs) deployed in a
month-long live show at the Edinburgh Festival Fringe. This case study
investigates human improvisers co-creating with conversational agents in a
professional theatre setting. We explore the technical capabilities and
constraints of on-the-spot multi-party dialogue, providing comprehensive
insights from both audience and performer experiences with AI on stage. Our
human-in-the-loop methodology underlines the challenges of these LLMs in
generating context-relevant responses, stressing the user interface's crucial
role. Audience feedback indicates an evolving interest for AI-driven live
entertainment, direct human-AI interaction, and a diverse range of expectations
about AI's conversational competence and utility as a creativity support tool.
Human performers express immense enthusiasm, varied satisfaction, and the
evolving public opinion highlights mixed emotions about AI's role in arts.
