  Curriculum reinforcement learning (CRL) improves the learning speed and
stability of an agent by exposing it to a tailored series of tasks throughout
learning. Despite empirical successes, an open question in CRL is how to
automatically generate a curriculum for a given reinforcement learning (RL)
agent, avoiding manual design. In this paper, we propose an answer by
interpreting the curriculum generation as an inference problem, where
distributions over tasks are progressively learned to approach the target task.
This approach leads to an automatic curriculum generation, whose pace is
controlled by the agent, with solid theoretical motivation and easily
integrated with deep RL algorithms. In the conducted experiments, the curricula
generated with the proposed algorithm significantly improve learning
performance across several environments and deep RL algorithms, matching or
outperforming state-of-the-art existing CRL algorithms.
