  Collocated tactile sensing is a fundamental enabling technology for dexterous
manipulation. However, deformable sensors introduce complex dynamics between
the robot, grasped object, and environment that must be considered for fine
manipulation. Here, we propose a method to learn soft tactile sensor membrane
dynamics that accounts for sensor deformations caused by the physical
interaction between the grasped object and environment. Our method combines the
perceived 3D geometry of the membrane with proprioceptive reaction wrenches to
predict future deformations conditioned on robot action. Grasped object poses
are recovered from membrane geometry and reaction wrenches, decoupling
interaction dynamics from the tactile observation model. We benchmark our
approach on two real-world contact-rich tasks: drawing with a grasped marker
and in-hand pivoting. Our results suggest that explicitly modeling membrane
dynamics achieves better task performance and generalization to unseen objects
than baselines.
