  We study the problem of semantic segmentation of large-scale 3D point clouds.
In recent years, significant research efforts have been directed toward local
feature aggregation, improved loss functions and sampling strategies. While the
fundamental framework of point cloud semantic segmentation has been largely
overlooked, with most existing approaches rely on the U-Net architecture by
default. In this paper, we propose U-Next, a small but mighty framework
designed for point cloud semantic segmentation. The key to this framework is to
learn multi-scale hierarchical representations from semantically similar
feature maps. Specifically, we build our U-Next by stacking multiple U-Net
$L^1$ codecs in a nested and densely arranged manner to minimize the semantic
gap, while simultaneously fusing the feature maps across scales to effectively
recover the fine-grained details. We also devised a multi-level deep
supervision mechanism to further smooth gradient propagation and facilitate
network optimization. Extensive experiments conducted on three large-scale
benchmarks including S3DIS, Toronto3D, and SensatUrban demonstrate the
superiority and the effectiveness of the proposed U-Next architecture. Our
U-Next architecture shows consistent and visible performance improvements
across different tasks and baseline models, indicating its great potential to
serve as a general framework for future research.
