  Reorienting diverse objects with a multi-fingered hand is a challenging task.
Current methods in robotic in-hand manipulation are either object-specific or
require permanent supervision of the object state from visual sensors. This is
far from human capabilities and from what is needed in real-world applications.
In this work, we address this gap by training shape-conditioned agents to
reorient diverse objects in hand, relying purely on tactile feedback (via
torque and position measurements of the fingers' joints). To achieve this, we
propose a learning framework that exploits shape information in a reinforcement
learning policy and a learned state estimator. We find that representing 3D
shapes by vectors from a fixed set of basis points to the shape's surface,
transformed by its predicted 3D pose, is especially helpful for learning
dexterous in-hand manipulation. In simulation and real-world experiments, we
show the reorientation of many objects with high success rates, on par with
state-of-the-art results obtained with specialized single-object agents.
Moreover, we show generalization to novel objects, achieving success rates of
$\sim$90% even for non-convex shapes.
