  Robotic object rearrangement combines the skills of picking and placing
objects. When object models are unavailable, typical collision-checking models
may be unable to predict collisions in partial point clouds with occlusions,
making generation of collision-free grasping or placement trajectories
challenging. We propose a learned collision model that accepts scene and query
object point clouds and predicts collisions for 6DOF object poses within the
scene. We train the model on a synthetic set of 1 million scene/object point
cloud pairs and 2 billion collision queries. We leverage the learned collision
model as part of a model predictive path integral (MPPI) policy in a tabletop
rearrangement task and show that the policy can plan collision-free grasps and
placements for objects unseen in training in both simulated and physical
cluttered scenes with a Franka Panda robot. The learned model outperforms both
traditional pipelines and learned ablations by 9.8% in accuracy on a dataset of
simulated collision queries and is 75x faster than the best-performing
baseline. Videos and supplementary material are available at
https://research.nvidia.com/publication/2021-03_Object-Rearrangement-Using.
