  Despite recent advances in Reinforcement Learning (RL), many problems,
especially real-world tasks, remain prohibitively expensive to learn. To
address this issue, several lines of research have explored how tasks, or data
samples themselves, can be sequenced into a curriculum to learn a problem that
may otherwise be too difficult to learn from scratch. However, generating and
optimizing a curriculum in a realistic scenario still requires extensive
interactions with the environment. To address this challenge, we formulate the
curriculum transfer problem, in which the schema of a curriculum optimized in a
simpler, easy-to-solve environment (e.g., a grid world) is transferred to a
complex, realistic scenario (e.g., a physics-based robotics simulation or the
real world). We present "ACuTE", Automatic Curriculum Transfer from Simple to
Complex Environments, a novel framework to solve this problem, and evaluate our
proposed method by comparing it to other baseline approaches (e.g., domain
adaptation) designed to speed up learning. We observe that our approach
produces improved jumpstart and time-to-threshold performance even when adding
task elements that further increase the difficulty of the realistic scenario.
Finally, we demonstrate that our approach is independent of the learning
algorithm used for curriculum generation, and is Sim2Real transferable to a
real world scenario using a physical robot.
