  Theatrical improvisation (impro or improv) is a demanding form of live,
collaborative performance. Improv is a humorous and playful artform built on an
open-ended narrative structure which simultaneously celebrates effort and
failure. It is thus an ideal test bed for the development and deployment of
interactive artificial intelligence (AI)-based conversational agents, or
artificial improvisors. This case study introduces an improv show experiment
featuring human actors and artificial improvisors. We have previously developed
a deep-learning-based artificial improvisor, trained on movie subtitles, that
can generate plausible, context-based, lines of dialogue suitable for theatre
(Mathewson and Mirowski 2017). In this work, we have employed it to control
what a subset of human actors say during an improv performance. We also give
human-generated lines to a different subset of performers. All lines are
provided to actors with headphones and all performers are wearing headphones.
This paper describes a Turing test, or imitation game, taking place in a
theatre, with both the audience members and the performers left to guess who is
a human and who is a machine. In order to test scientific hypotheses about the
perception of humans versus machines we collect anonymous feedback from
volunteer performers and audience members. Our results suggest that rehearsal
increases proficiency and possibility to control events in the performance.
That said, consistency with real world experience is limited by the interface
and the mechanisms used to perform the show. We also show that human-generated
lines are shorter, more positive, and have less difficult words with more
grammar and spelling mistakes than the artificial improvisor generated lines.
