  3D semantic segmentation is a fundamental building block for several scene
understanding applications such as autonomous driving, robotics and AR/VR.
Several state-of-the-art semantic segmentation models suffer from the part
misclassification problem, wherein parts of the same object are labelled
incorrectly. Previous methods have utilized hierarchical, iterative methods to
fuse semantic and instance information, but they lack learnability in context
fusion, and are computationally complex and heuristic driven. This paper
presents Segment-Fusion, a novel attention-based method for hierarchical fusion
of semantic and instance information to address the part misclassifications.
The presented method includes a graph segmentation algorithm for grouping
points into segments that pools point-wise features into segment-wise features,
a learnable attention-based network to fuse these segments based on their
semantic and instance features, and followed by a simple yet effective
connected component labelling algorithm to convert segment features to instance
labels. Segment-Fusion can be flexibly employed with any network architecture
for semantic/instance segmentation. It improves the qualitative and
quantitative performance of several semantic segmentation backbones by upto 5%
when evaluated on the ScanNet and S3DIS datasets.
