  Manipulating objects without grasping them enables more complex tasks, known
as non-prehensile manipulation. Most previous methods only learn one
manipulation skill, such as reach or push, and cannot achieve flexible object
manipulation.In this work, we introduce MRLM, a Multi-stage Reinforcement
Learning approach for non-prehensile Manipulation of objects.MRLM divides the
task into multiple stages according to the switching of object poses and
contact points.At each stage, the policy takes the point cloud-based state-goal
fusion representation as input, and proposes a spatially-continuous action that
including the motion of the parallel gripper pose and opening width.To fully
unlock the potential of MRLM, we propose a set of technical contributions
including the state-goal fusion representation, spatially-reachable distance
metric, and automatic buffer compaction.We evaluate MRLM on an Occluded
Grasping task which aims to grasp the object in configurations that are
initially occluded.Compared with the baselines, the proposed technical
contributions improve the success rate by at least 40\% and maximum 100\%, and
avoids falling into local optimum.Our method demonstrates strong generalization
to unseen object with shapes outside the training distribution.Moreover, MRLM
can be transferred to real world with zero-shot transfer, achieving a 95\%
success rate.Code and videos can be found at
https://sites.google.com/view/mrlm.
