  Complex and contact-rich robotic manipulation tasks, particularly those that
involve multi-fingered hands and underactuated object manipulation, present a
significant challenge to any control method. Methods based on reinforcement
learning offer an appealing choice for such settings, as they can enable robots
to learn to delicately balance contact forces and dexterously reposition
objects without strong modeling assumptions. However, running reinforcement
learning on real-world dexterous manipulation systems often requires
significant manual engineering. This negates the benefits of autonomous data
collection and ease of use that reinforcement learning should in principle
provide. In this paper, we describe a system for vision-based dexterous
manipulation that provides a "programming-free" approach for users to define
new tasks and enable robots with complex multi-fingered hands to learn to
perform them through interaction. The core principle underlying our system is
that, in a vision-based setting, users should be able to provide high-level
intermediate supervision that circumvents challenges in teleoperation or
kinesthetic teaching which allow a robot to not only learn a task efficiently
but also to autonomously practice. Our system includes a framework for users to
define a final task and intermediate sub-tasks with image examples, a
reinforcement learning procedure that learns the task autonomously without
interventions, and experimental results with a four-finger robotic hand
learning multi-stage object manipulation tasks directly in the real world,
without simulation, manual modeling, or reward engineering.
