  We propose a method that simultaneously estimates and controls extrinsic
contact with tactile feedback. The method enables challenging manipulation
tasks that require controlling light forces and accurate motions in contact,
such as balancing an unknown object on a thin rod standing upright. A factor
graph-based framework fuses a sequence of tactile and kinematic measurements to
estimate and control the interaction between gripper-object-environment,
including the location and wrench at the extrinsic contact between the grasped
object and the environment and the grasp wrench transferred from the gripper to
the object. The same framework simultaneously plans the gripper motions that
make it possible to estimate the state while satisfying regularizing control
objectives to prevent slip, such as minimizing the grasp wrench and minimizing
frictional force at the extrinsic contact. We show results with sub-millimeter
contact localization error and good slip prevention even on slippery
environments, for multiple contact formations (point, line, patch contact) and
transitions between them. See supplementary video and results at
https://sites.google.com/view/sim-tact.
