  We introduce a new setting of Novel Class Discovery in Semantic Segmentation
(NCDSS), which aims at segmenting unlabeled images containing new classes given
prior knowledge from a labeled set of disjoint classes. In contrast to existing
approaches that look at novel class discovery in image classification, we focus
on the more challenging semantic segmentation. In NCDSS, we need to distinguish
the objects and background, and to handle the existence of multiple classes
within an image, which increases the difficulty in using the unlabeled data. To
tackle this new setting, we leverage the labeled base data and a saliency model
to coarsely cluster novel classes for model training in our basic framework.
Additionally, we propose the Entropy-based Uncertainty Modeling and
Self-training (EUMS) framework to overcome noisy pseudo-labels, further
improving the model performance on the novel classes. Our EUMS utilizes an
entropy ranking technique and a dynamic reassignment to distill clean labels,
thereby making full use of the noisy data via self-supervised learning. We
build the NCDSS benchmark on the PASCAL-5$^i$ dataset and COCO-20$^i$ dataset.
Extensive experiments demonstrate the feasibility of the basic framework
(achieving an average mIoU of 49.81% on PASCAL-5$^i$) and the effectiveness of
EUMS framework (outperforming the basic framework by 9.28% mIoU on
PASCAL-5$^i$).
