  Model free reinforcement learning suffers from the high sampling complexity
inherent to robotic manipulation or locomotion tasks. Most successful
approaches typically use random sampling strategies which leads to slow policy
convergence. In this paper we present a novel approach for efficient
exploration that leverages previously learned tasks. We exploit the fact that
the same system is used across many tasks and build a generative model for
exploration based on data from previously solved tasks to improve learning new
tasks. The approach also enables continuous learning of improved exploration
strategies as novel tasks are learned. Extensive simulations on a robot
manipulator performing a variety of motion and contact interaction tasks
demonstrate the capabilities of the approach. In particular, our experiments
suggest that the exploration strategy can more than double learning speed,
especially when rewards are sparse. Moreover, the algorithm is robust to task
variations and parameter tuning, making it beneficial for complex robotic
problems.
