  Continual semantic segmentation aims to learn new classes while maintaining
the information from the previous classes. Although prior studies have shown
impressive progress in recent years, the fairness concern in the continual
semantic segmentation needs to be better addressed. Meanwhile, fairness is one
of the most vital factors in deploying the deep learning model, especially in
human-related or safety applications. In this paper, we present a novel
Fairness Continual Learning approach to the semantic segmentation problem. In
particular, under the fairness objective, a new fairness continual learning
framework is proposed based on class distributions. Then, a novel Prototypical
Contrastive Clustering loss is proposed to address the significant challenges
in continual learning, i.e., catastrophic forgetting and background shift. Our
proposed loss has also been proven as a novel, generalized learning paradigm of
knowledge distillation commonly used in continual learning. Moreover, the
proposed Conditional Structural Consistency loss further regularized the
structural constraint of the predicted segmentation. Our proposed approach has
achieved State-of-the-Art performance on three standard scene understanding
benchmarks, i.e., ADE20K, Cityscapes, and Pascal VOC, and promoted the fairness
of the segmentation model.
