  Training agents to autonomously learn how to use anthropomorphic robotic
hands has the potential to lead to systems capable of performing a multitude of
complex manipulation tasks in unstructured and uncertain environments. In this
work, we first introduce a suite of challenging simulated manipulation tasks
that current reinforcement learning and trajectory optimisation techniques find
difficult. These include environments where two simulated hands have to pass or
throw objects between each other, as well as an environment where the agent
must learn to spin a long pen between its fingers. We then introduce a simple
trajectory optimisation that performs significantly better than existing
methods on these environments. Finally, on the challenging PenSpin task we
combine sub-optimal demonstrations generated through trajectory optimisation
with off-policy reinforcement learning, obtaining performance that far exceeds
either of these approaches individually, effectively solving the environment.
Videos of all of our results are available at:
https://dexterous-manipulation.github.io/
