  Recent years have witnessed a great development of Convolutional Neural
Networks in semantic segmentation, where all classes of training images are
simultaneously available. In practice, new images are usually made available in
a consecutive manner, leading to a problem called Continual Semantic
Segmentation (CSS). Typically, CSS faces the forgetting problem since previous
training images are unavailable, and the semantic shift problem of the
background class. Considering the semantic segmentation as a context-dependent
pixel-level classification task, we explore CSS from a new perspective of
context analysis in this paper. We observe that the context of old-class pixels
in the new images is much more biased on new classes than that in the old
images, which can sharply aggravate the old-class forgetting and new-class
overfitting. To tackle the obstacle, we propose a biased-context-rectified CSS
framework with a context-rectified image-duplet learning scheme and a
biased-context-insensitive consistency loss. Furthermore, we propose an
adaptive re-weighting class-balanced learning strategy for the biased class
distribution. Our approach outperforms state-of-the-art methods by a large
margin in existing CSS scenarios.
