  In order to function in unstructured environments, robots need the ability to
recognize unseen objects. We take a step in this direction by tackling the
problem of segmenting unseen object instances in tabletop environments.
However, the type of large-scale real-world dataset required for this task
typically does not exist for most robotic settings, which motivates the use of
synthetic data. Our proposed method, UOIS-Net, separately leverages synthetic
RGB and synthetic depth for unseen object instance segmentation. UOIS-Net is
comprised of two stages: first, it operates only on depth to produce object
instance center votes in 2D or 3D and assembles them into rough initial masks.
Secondly, these initial masks are refined using RGB. Surprisingly, our
framework is able to learn from synthetic RGB-D data where the RGB is
non-photorealistic. To train our method, we introduce a large-scale synthetic
dataset of random objects on tabletops. We show that our method can produce
sharp and accurate segmentation masks, outperforming state-of-the-art methods
on unseen object instance segmentation. We also show that our method can
segment unseen objects for robot grasping.
