  We present MultiBodySync, a novel, end-to-end trainable multi-body motion
segmentation and rigid registration framework for multiple input 3D point
clouds. The two non-trivial challenges posed by this multi-scan multibody
setting that we investigate are: (i) guaranteeing correspondence and
segmentation consistency across multiple input point clouds capturing different
spatial arrangements of bodies or body parts; and (ii) obtaining robust
motion-based rigid body segmentation applicable to novel object categories. We
propose an approach to address these issues that incorporates spectral
synchronization into an iterative deep declarative network, so as to
simultaneously recover consistent correspondences as well as motion
segmentation. At the same time, by explicitly disentangling the correspondence
and motion segmentation estimation modules, we achieve strong generalizability
across different object categories. Our extensive evaluations demonstrate that
our method is effective on various datasets ranging from rigid parts in
articulated objects to individually moving objects in a 3D scene, be it
single-view or full point clouds.
