  Semantic segmentation has made striking progress due to the success of deep
convolutional neural networks. Considering the demands of autonomous driving,
real-time semantic segmentation has become a research hotspot these years.
However, few real-time RGB-D fusion semantic segmentation studies are carried
out despite readily accessible depth information nowadays. In this paper, we
propose a real-time fusion semantic segmentation network termed RFNet that
effectively exploits complementary cross-modal information. Building on an
efficient network architecture, RFNet is capable of running swiftly, which
satisfies autonomous vehicles applications. Multi-dataset training is leveraged
to incorporate unexpected small obstacle detection, enriching the recognizable
classes required to face unforeseen hazards in the real world. A comprehensive
set of experiments demonstrates the effectiveness of our framework. On
Cityscapes, Our method outperforms previous state-of-the-art semantic
segmenters, with excellent accuracy and 22Hz inference speed at the full
2048x1024 resolution, outperforming most existing RGB-D networks.
