  Curriculum design for reinforcement learning (RL) can speed up an agent's
learning process and help it learn to perform well on complex tasks. However,
existing techniques typically require domain-specific hyperparameter tuning,
involve expensive optimization procedures for task selection, or are suitable
only for specific learning objectives. In this work, we consider curriculum
design in contextual multi-task settings where the agent's final performance is
measured w.r.t. a target distribution over complex tasks. We base our
curriculum design on the Zone of Proximal Development concept, which has proven
to be effective in accelerating the learning process of RL agents for uniform
distribution over all tasks. We propose a novel curriculum, ProCuRL-Target,
that effectively balances the need for selecting tasks that are not too
difficult for the agent while progressing the agent's learning toward the
target distribution via leveraging task correlations. We theoretically justify
the task selection strategy of ProCuRL-Target by analyzing a simple learning
setting with REINFORCE learner model. Our experimental results across various
domains with challenging target task distributions affirm the effectiveness of
our curriculum strategy over state-of-the-art baselines in accelerating the
training process of deep RL agents.
