  Continuous in-hand manipulation is an important physical interaction skill,
where tactile sensing provides indispensable contact information to enable
dexterous manipulation of small objects. This work proposed a framework for
end-to-end policy learning with tactile feedback and sim-to-real transfer,
which achieved fine in-hand manipulation that controls the pose of a thin
cylindrical object, such as a long stick, to track various continuous
trajectories through multiple contacts of three fingertips of a dexterous robot
hand with tactile sensor arrays. We estimated the central contact position
between the stick and each fingertip from the high-dimensional tactile
information and showed that the learned policies achieved effective
manipulation performance with the processed tactile feedback. The policies were
trained with deep reinforcement learning in simulation and successfully
transferred to real-world experiments, using coordinated model calibration and
domain randomization. We evaluated the effectiveness of tactile information via
comparative studies and validated the sim-to-real performance through
real-world experiments.
