  This paper addresses the unrealistic aspect of the commonly adopted
Continuous Incremental Semantic Segmentation (CISS) scenario, termed
overlapped. We point out that overlapped allows the same image to reappear in
future tasks with different pixel labels, which is far from practical
incremental learning scenarios. Moreover, we identified that this flawed
scenario may lead to biased results for two commonly used techniques in CISS,
pseudo-labeling and exemplar memory, resulting in unintended advantages or
disadvantages for certain techniques. To mitigate this, a practical scenario
called partitioned is proposed, in which the dataset is first divided into
distinct subsets representing each class, and then the subsets are assigned to
each corresponding task. This efficiently addresses the issue above while
meeting the requirement of CISS scenario, such as capturing the background
shifts. Furthermore, we identify and address the code implementation issues
related to retrieving data from the exemplar memory, which was ignored in
previous works. Lastly, we introduce a simple yet competitive memory-based
baseline, MiB-AugM, that handles background shifts of current tasks in the
exemplar memory. This baseline achieves state-of-the-art results across
multiple tasks involving learning numerous new classes.
