  Applications in fields ranging from home care to warehouse fulfillment to
surgical assistance require robots to reliably manipulate the shape of 3D
deformable objects. Analytic models of elastic, 3D deformable objects require
numerous parameters to describe the potentially infinite degrees of freedom
present in determining the object's shape. Previous attempts at performing 3D
shape control rely on hand-crafted features to represent the object shape and
require training of object-specific control models. We overcome these issues
through the use of our novel DeformerNet neural network architecture, which
operates on a partial-view point cloud of the manipulated object and a point
cloud of the goal shape to learn a low-dimensional representation of the object
shape. This shape embedding enables the robot to learn a visual servo
controller that computes the desired robot end-effector action to iteratively
deform the object toward the target shape. We demonstrate both in simulation
and on a physical robot that DeformerNet reliably generalizes to object shapes
and material stiffness not seen during training, including ex vivo chicken
muscle tissue. Crucially, using DeformerNet, the robot successfully
accomplishes three surgical sub-tasks: retraction (moving tissue aside to
access a site underneath it), tissue wrapping (a sub-task in procedures like
aortic stent placements), and connecting two tubular pieces of tissue (a
sub-task in anastomosis).
