  Current point cloud semantic segmentation has achieved great advances when
given sufficient labels. However, the dense annotation of LiDAR point clouds
remains prohibitively expensive and time-consuming, unable to keep up with the
continuously growing volume of data. In this paper, we propose annotating
images with scattered points, followed by utilizing SAM (a Foundation model) to
generate semantic segmentation labels for the images. Finally, by mapping the
segmentation labels of the images to the LiDAR space using the intrinsic and
extrinsic parameters of the camera and LiDAR, we obtain labels for point cloud
semantic segmentation, and release Scatter-KITTI and Scatter-nuScenes, which
are the first works to utilize image segmentation-based SAM for weakly
supervised point cloud semantic segmentation. Furthermore, to mitigate the
influence of erroneous pseudo labels obtained from sparse annotations on point
cloud features, we propose a multi-modal weakly supervised network for LiDAR
semantic segmentation, called MM-ScatterNet. This network combines features
from both point cloud and image modalities, enhancing the representation
learning of point clouds by introducing consistency constraints between
multi-modal features and point cloud features. On the SemanticKITTI dataset, we
achieve 66\% of fully supervised performance using only 0.02% of annotated
data, and on the NuScenes dataset, we achieve 95% of fully supervised
performance using only 0.1% labeled points.
