  Learning policies for complex tasks that require multiple different skills is
a major challenge in reinforcement learning (RL). It is also a requirement for
its deployment in real-world scenarios. This paper proposes a novel framework
for efficient multi-task reinforcement learning. Our framework trains agents to
employ hierarchical policies that decide when to use a previously learned
policy and when to learn a new skill. This enables agents to continually
acquire new skills during different stages of training. Each learned task
corresponds to a human language description. Because agents can only access
previously learned skills through these descriptions, the agent can always
provide a human-interpretable description of its choices. In order to help the
agent learn the complex temporal dependencies necessary for the hierarchical
policy, we provide it with a stochastic temporal grammar that modulates when to
rely on previously learned skills and when to execute new skills. We validate
our approach on Minecraft games designed to explicitly test the ability to
reuse previously learned skills while simultaneously learning new skills.
