  Open-ended worlds are those in which there are no pre-specified goals or
environmental reward signal. As a consequence, an agent must know how to
perform a multitude of tasks. However, when a new task is presented to an
agent, we expect it to be able to reuse some of what it knows from previous
tasks to rapidly learn that new task. We introduce a novel technique whereby
policies for different a priori known tasks are combined into a
Mixture-of-Experts model with an attention mechanism across a mix of frozen and
unfrozen experts. The model learns when to attend to frozen task-specific
experts when appropriate and learns new experts to handle novel situations. We
work in an open-ended text-based environment in which the agent is tasked with
behaving like different types of character roles and must rapidly learn
behaviors associated with new character role types. We show that our agent both
obtains more rewards in the zero-shot setting, and discovers these rewards with
greater sample efficiency in the few-shot learning settings.
