  The exploration of robotic dexterous hands utilizing tools has recently
attracted considerable attention. A significant challenge in this field is the
precise awareness of a tool's pose when grasped, as occlusion by the hand often
degrades the quality of the estimation. Additionally, the tool's overall pose
often fails to accurately represent the contact interaction, thereby limiting
the effectiveness of vision-guided, contact-dependent activities. To overcome
this limitation, we present the innovative TOOLEE dataset, which, to the best
of our knowledge, is the first to feature affordance segmentation of a tool's
end-effector (EE) along with its defined 6D pose based on its usage.
Furthermore, we propose the ToolEENet framework for accurate 6D pose estimation
of the tool's EE. This framework begins by segmenting the tool's EE from raw
RGBD data, then uses a diffusion model-based pose estimator for 6D pose
estimation at a category-specific level. Addressing the issue of symmetry in
pose estimation, we introduce a symmetry-aware pose representation that
enhances the consistency of pose estimation. Our approach excels in this field,
demonstrating high levels of precision and generalization. Furthermore, it
shows great promise for application in contact-based manipulation scenarios.
All data and codes are available on the project website:
https://yuyangtu.github.io/projectToolEENet.html
