  In this paper, we consider the problem of non-prehensile manipulation using
grasped objects. This problem is a superset of many common manipulation skills
including instances of tool-use (e.g., grasped spatula flipping a burger) and
assembly (e.g., screwdriver tightening a screw). Here, we present an
algorithmic approach for non-prehensile manipulation leveraging a gripper with
highly compliant and high-resolution tactile sensors. Our approach solves for
robot actions that drive object poses and forces to desired values while
obeying the complex dynamics induced by the sensors as well as the constraints
imposed by static equilibrium, object kinematics, and frictional contact. Our
method is able to produce a variety of manipulation skills and is amenable to
gradient-based optimization by exploiting differentiability within contact
modes (e.g., specifications of sticking or sliding contacts). We evaluate 4
variants of controllers that attempt to realize these plans and demonstrate a
number of complex skills including non-prehensile planar sliding and pivoting
on a variety of object geometries. The perception and controls capabilities
that drive these skills are the building blocks towards dexterous and reactive
autonomy in unstructured environments.
