  Critical research about camera-and-LiDAR-based semantic object segmentation
for autonomous driving significantly benefited from the recent development of
deep learning. Specifically, the vision transformer is the novel ground-breaker
that successfully brought the multi-head-attention mechanism to computer vision
applications. Therefore, we propose a vision-transformer-based network to carry
out camera-LiDAR fusion for semantic segmentation applied to autonomous
driving. Our proposal uses the novel progressive-assemble strategy of vision
transformers on a double-direction network and then integrates the results in a
cross-fusion strategy over the transformer decoder layers. Unlike other works
in the literature, our camera-LiDAR fusion transformers have been evaluated in
challenging conditions like rain and low illumination, showing robust
performance. The paper reports the segmentation results over the vehicle and
human classes in different modalities: camera-only, LiDAR-only, and
camera-LiDAR fusion. We perform coherent controlled benchmark experiments of
CLFT against other networks that are also designed for semantic segmentation.
The experiments aim to evaluate the performance of CLFT independently from two
perspectives: multimodal sensor fusion and backbone architectures. The
quantitative assessments show our CLFT networks yield an improvement of up to
10% for challenging dark-wet conditions when comparing with
Fully-Convolutional-Neural-Network-based (FCN) camera-LiDAR fusion neural
network. Contrasting to the network with transformer backbone but using single
modality input, the all-around improvement is 5-10%.
