  Continual semantic segmentation (CSS) based on incremental learning (IL) is a
great endeavour in developing human-like segmentation models. However, current
CSS approaches encounter challenges in the trade-off between preserving old
knowledge and learning new ones, where they still need large-scale annotated
data for incremental training and lack interpretability. In this paper, we
present Learning at a Glance (LAG), an efficient, robust, human-like and
interpretable approach for CSS. Specifically, LAG is a simple and
model-agnostic architecture, yet it achieves competitive CSS efficiency with
limited incremental data. Inspired by human-like recognition patterns, we
propose a semantic-invariance modelling approach via semantic features
decoupling that simultaneously reconciles solid knowledge inheritance and
new-term learning. Concretely, the proposed decoupling manner includes two
ways, i.e., channel-wise decoupling and spatial-level neuron-relevant semantic
consistency. Our approach preserves semantic-invariant knowledge as solid
prototypes to alleviate catastrophic forgetting, while also constraining
sample-specific contents through an asymmetric contrastive learning method to
enhance model robustness during IL steps. Experimental results in multiple
datasets validate the effectiveness of the proposed method. Furthermore, we
introduce a novel CSS protocol that better reflects realistic data-limited CSS
settings, and LAG achieves superior performance under multiple data-limited
conditions.
