  Large pre-trained neural language models (LM) have very powerful text
generation capabilities. However, in practice, they are hard to control for
creative purposes. We describe a Plug-and-Play controllable language generation
framework, Plug-and-Blend, that allows a human user to input multiple control
codes (topics). In the context of automated story generation, this allows a
human user loose or fine-grained control of the topics and transitions between
them that will appear in the generated story, and can even allow for
overlapping, blended topics. Automated evaluations show our framework, working
with different generative LMs, controls the generation towards given
continuous-weighted control codes while keeping the generated sentences fluent,
demonstrating strong blending capability. A human participant evaluation shows
that the generated stories are observably transitioning between two topics.
