  3D point cloud semantic segmentation has a wide range of applications.
Recently, weakly supervised point cloud segmentation methods have been
proposed, aiming to alleviate the expensive and laborious manual annotation
process by leveraging scene-level labels. However, these methods have not
effectively exploited the rich geometric information (such as shape and scale)
and appearance information (such as color and texture) present in RGB-D scans.
Furthermore, current approaches fail to fully leverage the point affinity that
can be inferred from the feature extraction network, which is crucial for
learning from weak scene-level labels. Additionally, previous work overlooks
the detrimental effects of the long-tailed distribution of point cloud data in
weakly supervised 3D semantic segmentation. To this end, this paper proposes a
simple yet effective scene-level weakly supervised point cloud segmentation
method with a newly introduced multi-modality point affinity inference module.
The point affinity proposed in this paper is characterized by features from
multiple modalities (e.g., point cloud and RGB), and is further refined by
normalizing the classifier weights to alleviate the detrimental effects of
long-tailed distribution without the need of the prior of category
distribution. Extensive experiments on the ScanNet and S3DIS benchmarks verify
the effectiveness of our proposed method, which outperforms the
state-of-the-art by ~4% to ~6% mIoU. Codes are released at
https://github.com/Sunny599/AAAI24-3DWSSG-MMA.
