  Grasping an object when it is in an ungraspable pose is a challenging task,
such as books or other large flat objects placed horizontally on a table.
Inspired by human manipulation, we address this problem by pushing the object
to the edge of the table and then grasping it from the hanging part. In this
paper, we develop a model-free Deep Reinforcement Learning framework to
synergize pushing and grasping actions. We first pre-train a Variational
Autoencoder to extract high-dimensional features of input scenario images. One
Proximal Policy Optimization algorithm with the common reward and sharing
layers of Actor-Critic is employed to learn both pushing and grasping actions
with high data efficiency. Experiments show that our one network policy can
converge 2.5 times faster than the policy using two parallel networks.
Moreover, the experiments on unseen objects show that our policy can generalize
to the challenging case of objects with curved surfaces and off-center
irregularly shaped objects. Lastly, our policy can be transferred to a real
robot without fine-tuning by using CycleGAN for domain adaption and outperforms
the push-to-wall baseline.
