  The Reinforcement Learning (RL) paradigm has been an essential tool for
automating robotic tasks. Despite the advances in RL, it is still not widely
adopted in the industry due to the need for an expensive large amount of robot
interaction with its environment. Curriculum Learning (CL) has been proposed to
expedite learning. However, most research works have been only evaluated in
simulated environments, from video games to robotic toy tasks. This paper
presents a study for accelerating robot learning of contact-rich manipulation
tasks based on Curriculum Learning combined with Domain Randomization (DR). We
tackle complex industrial assembly tasks with position-controlled robots, such
as insertion tasks. We compare different curricula designs and sampling
approaches for DR. Based on this study, we propose a method that significantly
outperforms previous work, which uses DR only (No CL is used), with less than a
fifth of the training time (samples). Results also show that even when training
only in simulation with toy tasks, our method can learn policies that can be
transferred to the real-world robot. The learned policies achieved success
rates of up to 86\% on real-world complex industrial insertion tasks (with
tolerances of $\pm 0.01~mm$) not seen during the training.
