  In-hand tool manipulation is an operation that not only manipulates a tool
within the hand (i.e., in-hand manipulation) but also achieves a grasp suitable
for a task after the manipulation. This study aims to achieve an in-hand tool
manipulation skill through deep reinforcement learning. The difficulty of
learning the skill arises because this manipulation requires (A) exploring
long-term contact-state changes to achieve the desired grasp and (B)
highly-varied motions depending on the contact-state transition. (A) leads to a
sparsity of a reward on a successful grasp, and (B) requires an RL agent to
explore widely within the state-action space to learn highly-varied actions,
leading to sample inefficiency. To address these issues, this study proposes
Action Primitives based on Contact-state Transition (APriCoT). APriCoT
decomposes the manipulation into short-term action primitives by describing the
operation as a contact-state transition based on three action representations
(detach, crossover, attach). In each action primitive, fingers are required to
perform short-term and similar actions. By training a policy for each
primitive, we can mitigate the issues from (A) and (B). This study focuses on a
fundamental operation as an example of in-hand tool manipulation: rotating an
elongated object grasped with a precision grasp by half a turn to achieve the
initial grasp. Experimental results demonstrated that ours succeeded in both
the rotation and the achievement of the desired grasp, unlike existing studies.
Additionally, it was found that the policy was robust to changes in object
shape.
