  Continual Learning in semantic scene segmentation aims to continually learn
new unseen classes in dynamic environments while maintaining previously learned
knowledge. Prior studies focused on modeling the catastrophic forgetting and
background shift challenges in continual learning. However, fairness, another
major challenge that causes unfair predictions leading to low performance among
major and minor classes, still needs to be well addressed. In addition, prior
methods have yet to model the unknown classes well, thus resulting in producing
non-discriminative features among unknown classes. This paper presents a novel
Fairness Learning via Contrastive Attention Approach to continual learning in
semantic scene understanding. In particular, we first introduce a new Fairness
Contrastive Clustering loss to address the problems of catastrophic forgetting
and fairness. Then, we propose an attention-based visual grammar approach to
effectively model the background shift problem and unknown classes, producing
better feature representations for different unknown classes. Through our
experiments, our proposed approach achieves State-of-the-Art (SOTA) performance
on different continual learning benchmarks, i.e., ADE20K, Cityscapes, and
Pascal VOC. It promotes the fairness of the continual semantic segmentation
model.
