  We present the first real-time system capable of tracking and reconstructing,
individually, every visible object in a given scene, without any form of prior
on the rigidness of the objects, texture existence, or object category. In
contrast with previous methods such as Co-Fusion and MaskFusion that first
segment the scene into individual objects and then process each object
independently, the proposed method dynamically segments the non-rigid scene as
part of the tracking and reconstruction process. When new measurements indicate
topology change, reconstructed models are updated in real-time to reflect that
change. Our proposed system can provide the live geometry and deformation of
all visible objects in a novel scene in real-time, which makes it possible to
be integrated seamlessly into numerous existing robotics applications that rely
on object models for grasping and manipulation. The capabilities of the
proposed system are demonstrated in challenging scenes that contain multiple
rigid and non-rigid objects.
