  Manipulation tasks such as preparing a meal or assembling furniture remain
highly challenging for robotics and vision. Traditional task and motion
planning (TAMP) methods can solve complex tasks but require full state
observability and are not adapted to dynamic scene changes. Recent learning
methods can operate directly on visual inputs but typically require many
demonstrations and/or task-specific reward engineering. In this work we aim to
overcome previous limitations and propose a reinforcement learning (RL)
approach to task planning that learns to combine primitive skills. First,
compared to previous learning methods, our approach requires neither
intermediate rewards nor complete task demonstrations during training. Second,
we demonstrate the versatility of our vision-based task planning in challenging
settings with temporary occlusions and dynamic scene changes. Third, we propose
an efficient training of basic skills from few synthetic demonstrations by
exploring recent CNN architectures and data augmentation. Notably, while all of
our policies are learned on visual inputs in simulated environments, we
demonstrate the successful transfer and high success rates when applying such
policies to manipulation tasks on a real UR5 robotic arm.
