  We extend panoptic segmentation to the open-world and introduce an open-set
panoptic segmentation (OPS) task. This task requires performing panoptic
segmentation for not only known classes but also unknown ones that have not
been acknowledged during training. We investigate the practical challenges of
the task and construct a benchmark on top of an existing dataset, COCO. In
addition, we propose a novel exemplar-based open-set panoptic segmentation
network (EOPSN) inspired by exemplar theory. Our approach identifies a new
class based on exemplars, which are identified by clustering and employed as
pseudo-ground-truths. The size of each class increases by mining new exemplars
based on the similarities to the existing ones associated with the class. We
evaluate EOPSN on the proposed benchmark and demonstrate the effectiveness of
our proposals. The primary goal of our work is to draw the attention of the
community to the recognition in the open-world scenarios. The implementation of
our algorithm is available on the project webpage:
https://cv.snu.ac.kr/research/EOPSN.
