  In this paper, we propose a novel approach to 3D deformable object
manipulation leveraging a deep neural network called DeformerNet. Controlling
the shape of a 3D object requires an effective state representation that can
capture the full 3D geometry of the object. Current methods work around this
problem by defining a set of feature points on the object or only deforming the
object in 2D image space, which does not truly address the 3D shape control
problem. Instead, we explicitly use 3D point clouds as the state representation
and apply Convolutional Neural Network on point clouds to learn the 3D
features. These features are then mapped to the robot end-effector's position
using a fully-connected neural network. Once trained in an end-to-end fashion,
DeformerNet directly maps the current point cloud of a deformable object, as
well as a target point cloud shape, to the desired displacement in robot
gripper position. In addition, we investigate the problem of predicting the
manipulation point location given the initial and goal shape of the object.
