  We present ClusterVO, a stereo Visual Odometry which simultaneously clusters
and estimates the motion of both ego and surrounding rigid clusters/objects.
Unlike previous solutions relying on batch input or imposing priors on scene
structure or dynamic object models, ClusterVO is online, general and thus can
be used in various scenarios including indoor scene understanding and
autonomous driving. At the core of our system lies a multi-level probabilistic
association mechanism and a heterogeneous Conditional Random Field (CRF)
clustering approach combining semantic, spatial and motion information to
jointly infer cluster segmentations online for every frame. The poses of camera
and dynamic objects are instantly solved through a sliding-window optimization.
Our system is evaluated on Oxford Multimotion and KITTI dataset both
quantitatively and qualitatively, reaching comparable results to
state-of-the-art solutions on both odometry and dynamic trajectory recovery.
