  Modern incremental learning for semantic segmentation methods usually learn
new categories based on dense annotations. Although achieve promising results,
pixel-by-pixel labeling is costly and time-consuming. Weakly incremental
learning for semantic segmentation (WILSS) is a novel and attractive task,
which aims at learning to segment new classes from cheap and widely available
image-level labels. Despite the comparable results, the image-level labels can
not provide details to locate each segment, which limits the performance of
WILSS. This inspires us to think how to improve and effectively utilize the
supervision of new classes given image-level labels while avoiding forgetting
old ones. In this work, we propose a novel and data-efficient framework for
WILSS, named FMWISS. Specifically, we propose pre-training based
co-segmentation to distill the knowledge of complementary foundation models for
generating dense pseudo labels. We further optimize the noisy pseudo masks with
a teacher-student architecture, where a plug-in teacher is optimized with a
proposed dense contrastive loss. Moreover, we introduce memory-based copy-paste
augmentation to improve the catastrophic forgetting problem of old classes.
Extensive experiments on Pascal VOC and COCO datasets demonstrate the superior
performance of our framework, e.g., FMWISS achieves 70.7% and 73.3% in the 15-5
VOC setting, outperforming the state-of-the-art method by 3.4% and 6.1%,
respectively.
