  Semantic segmentation methods can not directly identify abnormal objects in
images. Anomaly Segmentation algorithm from this realistic setting can
distinguish between in-distribution objects and Out-Of-Distribution (OOD)
objects and output the anomaly probability for pixels. In this paper, a
Prototype Guided Anomaly segmentation Network (PGAN) is proposed to extract
semantic prototypes for in-distribution training data from limited annotated
images. In the model, prototypes are used to model the hierarchical category
semantic information and distinguish OOD pixels. The proposed PGAN model
includes a semantic segmentation network and a prototype extraction network.
Similarity measures are adopted to optimize the prototypes. The learned
semantic prototypes are used as category semantics to compare the similarity
with features extracted from test images and then to generate semantic
segmentation prediction. The proposed prototype extraction network can also be
integrated into most semantic segmentation networks and recognize OOD pixels.
On the StreetHazards dataset, the proposed PGAN model produced mIoU of 53.4%
for anomaly segmentation. The experimental results demonstrate PGAN may achieve
the SOTA performance in the anomaly segmentation tasks.
