  Curriculum learning in reinforcement learning is a training methodology that
seeks to speed up learning of a difficult target task, by first training on a
series of simpler tasks and transferring the knowledge acquired to the target
task. Automatically choosing a sequence of such tasks (i.e. a curriculum) is an
open problem that has been the subject of much recent work in this area. In
this paper, we build upon a recent method for curriculum design, which
formulates the curriculum sequencing problem as a Markov Decision Process. We
extend this model to handle multiple transfer learning algorithms, and show for
the first time that a curriculum policy over this MDP can be learned from
experience. We explore various representations that make this possible, and
evaluate our approach by learning curriculum policies for multiple agents in
two different domains. The results show that our method produces curricula that
can train agents to perform on a target task as fast or faster than existing
methods.
