  Although end-to-end robot learning has shown some success for robot
manipulation, the learned policies are often not sufficiently robust to
variations in object pose or geometry. To improve the policy generalization, we
introduce spatially-grounded parameterized motion primitives in our method
HACMan++. Specifically, we propose an action representation consisting of three
components: what primitive type (such as grasp or push) to execute, where the
primitive will be grounded (e.g. where the gripper will make contact with the
world), and how the primitive motion is executed, such as parameters specifying
the push direction or grasp orientation. These three components define a novel
discrete-continuous action space for reinforcement learning. Our framework
enables robot agents to learn to chain diverse motion primitives together and
select appropriate primitive parameters to complete long-horizon manipulation
tasks. By grounding the primitives on a spatial location in the environment,
our method is able to effectively generalize across object shape and pose
variations. Our approach significantly outperforms existing methods,
particularly in complex scenarios demanding both high-level sequential
reasoning and object generalization. With zero-shot sim-to-real transfer, our
policy succeeds in challenging real-world manipulation tasks, with
generalization to unseen objects. Videos can be found on the project website:
https://sgmp-rss2024.github.io.
