  The goal of incremental Few-shot Semantic Segmentation (iFSS) is to extend
pre-trained segmentation models to new classes via few annotated images without
access to old training data. During incrementally learning novel classes, the
data distribution of old classes will be destroyed, leading to catastrophic
forgetting. Meanwhile, the novel classes have only few samples, making models
impossible to learn the satisfying representations of novel classes. For the
iFSS problem, we propose a network called OINet, i.e., the background embedding
space \textbf{O}rganization and prototype \textbf{I}nherit Network.
Specifically, when training base classes, OINet uses multiple classification
heads for the background and sets multiple sub-class prototypes to reserve
embedding space for the latent novel classes. During incrementally learning
novel classes, we propose a strategy to select the sub-class prototypes that
best match the current learning novel classes and make the novel classes
inherit the selected prototypes' embedding space. This operation allows the
novel classes to be registered in the embedding space using few samples without
affecting the distribution of the base classes. Results on Pascal-VOC and COCO
show that OINet achieves a new state of the art.
