  Camera and 3D LiDAR sensors have become indispensable devices in modern
autonomous driving vehicles, where the camera provides the fine-grained
texture, color information in 2D space and LiDAR captures more precise and
farther-away distance measurements of the surrounding environments. The
complementary information from these two sensors makes the two-modality fusion
be a desired option. However, two major issues of the fusion between camera and
LiDAR hinder its performance, \ie, how to effectively fuse these two modalities
and how to precisely align them (suffering from the weak spatiotemporal
synchronization problem). In this paper, we propose a coarse-to-fine LiDAR and
camera fusion-based network (termed as LIF-Seg) for LiDAR segmentation. For the
first issue, unlike these previous works fusing the point cloud and image
information in a one-to-one manner, the proposed method fully utilizes the
contextual information of images and introduces a simple but effective
early-fusion strategy. Second, due to the weak spatiotemporal synchronization
problem, an offset rectification approach is designed to align these
two-modality features. The cooperation of these two components leads to the
success of the effective camera-LiDAR fusion. Experimental results on the
nuScenes dataset show the superiority of the proposed LIF-Seg over existing
methods with a large margin. Ablation studies and analyses demonstrate that our
proposed LIF-Seg can effectively tackle the weak spatiotemporal synchronization
problem.
