  Transfer learning can speed up training in machine learning and is regularly
used in classification tasks. It reuses prior knowledge from other tasks to
pre-train networks for new tasks. In reinforcement learning, learning actions
for a behavior policy that can be applied to new environments is still a
challenge, especially for tasks that involve much planning. Sokoban is a
challenging puzzle game. It has been used widely as a benchmark in
planning-based reinforcement learning. In this paper, we show how prior
knowledge improves learning in Sokoban tasks. We find that reusing feature
representations learned previously can accelerate learning new, more complex,
instances. In effect, we show how curriculum learning, from simple to complex
tasks, works in Sokoban. Furthermore, feature representations learned in
simpler instances are more general, and thus lead to positive transfers towards
more complex tasks, but not vice versa. We have also studied which part of the
knowledge is most important for transfer to succeed, and identify which layers
should be used for pre-training.
