  We study the problem of object retrieval in scenarios where visual sensing is
absent, object shapes are unknown beforehand and objects can move freely, like
grabbing objects out of a drawer. Successful solutions require localizing free
objects, identifying specific object instances, and then grasping the
identified objects, only using touch feedback. Unlike vision, where cameras can
observe the entire scene, touch sensors are local and only observe parts of the
scene that are in contact with the manipulator. Moreover, information gathering
via touch sensors necessitates applying forces on the touched surface which may
disturb the scene itself. Reasoning with touch, therefore, requires careful
exploration and integration of information over time -- a challenge we tackle.
We present a system capable of using sparse tactile feedback from fingertip
touch sensors on a dexterous hand to localize, identify and grasp novel objects
without any visual feedback. Videos are available at
https://taochenshh.github.io/projects/tactofind.
