  Robotic manipulation can benefit from wrist-mounted force/torque (F/T)
sensors, but conventional F/T sensors can be expensive, difficult to install,
and damaged by high loads. We present Visual Force/Torque Sensing (VFTS), a
method that visually estimates the 6-axis F/T measurement that would be
reported by a conventional F/T sensor. In contrast to approaches that sense
loads using internal cameras placed behind soft exterior surfaces, our approach
uses an external camera with a fisheye lens that observes a soft gripper. VFTS
includes a deep learning model that takes a single RGB image as input and
outputs a 6-axis F/T estimate. We trained the model with sensor data collected
while teleoperating a robot (Stretch RE1 from Hello Robot Inc.) to perform
manipulation tasks. VFTS outperformed F/T estimates based on motor currents,
generalized to a novel home environment, and supported three autonomous tasks
relevant to healthcare: grasping a blanket, pulling a blanket over a manikin,
and cleaning a manikin's limbs. VFTS also performed well with a manually
operated pneumatic gripper. Overall, our results suggest that an external
camera observing a soft gripper can perform useful visual force/torque sensing
for a variety of manipulation tasks.
