  Touch sensing is widely acknowledged to be important for dexterous robotic
manipulation, but exploiting tactile sensing for continuous, non-prehensile
manipulation is challenging. General purpose control techniques that are able
to effectively leverage tactile sensing as well as accurate physics models of
contacts and forces remain largely elusive, and it is unclear how to even
specify a desired behavior in terms of tactile percepts. In this paper, we take
a step towards addressing these issues by combining high-resolution tactile
sensing with data-driven modeling using deep neural network dynamics models. We
propose deep tactile MPC, a framework for learning to perform tactile servoing
from raw tactile sensor inputs, without manual supervision. We show that this
method enables a robot equipped with a GelSight-style tactile sensor to
manipulate a ball, analog stick, and 20-sided die, learning from unsupervised
autonomous interaction and then using the learned tactile predictive model to
reposition each object to user-specified configurations, indicated by a goal
tactile reading. Videos, visualizations and the code are available here:
https://sites.google.com/view/deeptactilempc
