  Classification of different object surface material types can play a
significant role in the decision-making algorithms for mobile robots and
autonomous vehicles. RGB-based scene-level semantic segmentation has been
well-addressed in the literature. However, improving material recognition using
the depth modality and its integration with SLAM algorithms for 3D semantic
mapping could unlock new potential benefits in the robotics perception
pipeline. To this end, we propose a complementarity-aware deep learning
approach for RGB-D-based material classification built on top of an
object-oriented pipeline. The approach further integrates the ORB-SLAM2 method
for 3D scene mapping with multiscale clustering of the detected material
semantics in the point cloud map generated by the visual SLAM algorithm.
Extensive experimental results with existing public datasets and newly
contributed real-world robot datasets demonstrate a significant improvement in
material classification and 3D clustering accuracy compared to state-of-the-art
approaches for 3D semantic scene mapping.
