  Semantic segmentation and semantic edge detection can be seen as two dual
problems with close relationships in computer vision. Despite the fast
evolution of learning-based 3D semantic segmentation methods, little attention
has been drawn to the learning of 3D semantic edge detectors, even less to a
joint learning method for the two tasks. In this paper, we tackle the 3D
semantic edge detection task for the first time and present a new two-stream
fully-convolutional network that jointly performs the two tasks. In particular,
we design a joint refinement module that explicitly wires region information
and edge information to improve the performances of both tasks. Further, we
propose a novel loss function that encourages the network to produce semantic
segmentation results with better boundaries. Extensive evaluations on S3DIS and
ScanNet datasets show that our method achieves on par or better performance
than the state-of-the-art methods for semantic segmentation and outperforms the
baseline methods for semantic edge detection. Code release:
https://github.com/hzykent/JSENet
