  In order to deploy autonomous agents to domains such as autonomous driving,
infrastructure management, health care, and finance, they must be able to adapt
safely to unseen situations. The current approach in constructing such agents
is to try to include as much variation into training as possible, and then
generalize within the possible variations. This paper proposes a principled
approach where a context module is coevolved with a skill module. The context
module recognizes the variation and modulates the skill module so that the
entire system performs well in unseen situations. The approach is evaluated in
a challenging version of the Flappy Bird game where the effects of the actions
vary over time. The Context+Skill approach leads to significantly more robust
behavior in environments with previously unseen effects. Such a principled
generalization ability is essential in deploying autonomous agents in real
world tasks, and can serve as a foundation for continual learning as well.
