  In order to deploy autonomous agents in digital interactive environments,
they must be able to act robustly in unseen situations. The standard machine
learning approach is to include as much variation as possible into training
these agents. The agents can then interpolate within their training, but they
cannot extrapolate much beyond it. This paper proposes a principled approach
where a context module is coevolved with a skill module in the game. The
context module recognizes the temporal variation in the game and modulates the
outputs of the skill module so that the action decisions can be made robustly
even in previously unseen situations. The approach is evaluated in the Flappy
Bird and LunarLander video games, as well as in the CARLA autonomous driving
simulation. The Context+Skill approach leads to significantly more robust
behavior in environments that require extrapolation beyond training. Such a
principled generalization ability is essential in deploying autonomous agents
in real-world tasks, and can serve as a foundation for continual adaptation as
well.
