  The 3D scene understanding is mainly considered as a crucial requirement in
computer vision and robotics applications. One of the high-level tasks in 3D
scene understanding is semantic segmentation of RGB-Depth images. With the
availability of RGB-D cameras, it is desired to improve the accuracy of the
scene understanding process by exploiting the depth features along with the
appearance features. As depth images are independent of illumination, they can
improve the quality of semantic labeling alongside RGB images. Consideration of
both common and specific features of these two modalities improves the
performance of semantic segmentation. One of the main problems in RGB-Depth
semantic segmentation is how to fuse or combine these two modalities to achieve
more advantages of each modality while being computationally efficient.
Recently, the methods that encounter deep convolutional neural networks have
reached the state-of-the-art results by early, late, and middle fusion
strategies. In this paper, an efficient encoder-decoder model with the
attention-based fusion block is proposed to integrate mutual influences between
feature maps of these two modalities. This block explicitly extracts the
interdependences among concatenated feature maps of these modalities to exploit
more powerful feature maps from RGB-Depth images. The extensive experimental
results on three main challenging datasets of NYU-V2, SUN RGB-D, and Stanford
2D-3D-Semantic show that the proposed network outperforms the state-of-the-art
models with respect to computational cost as well as model size. Experimental
results also illustrate the effectiveness of the proposed lightweight
attention-based fusion model in terms of accuracy.
