  Due to the model aging problem, Deep Neural Networks (DNNs) need updates to
adjust them to new data distributions. The common practice leverages
incremental learning (IL), e.g., Class-based Incremental Learning (CIL) that
updates output labels, to update the model with new data and a limited number
of old data. This avoids heavyweight training (from scratch) using conventional
methods and saves storage space by reducing the number of old data to store.
But it also leads to poor performance in fairness. In this paper, we show that
CIL suffers both dataset and algorithm bias problems, and existing solutions
can only partially solve the problem. We propose a novel framework, CILIATE,
that fixes both dataset and algorithm bias in CIL. It features a novel
differential analysis guided dataset and training refinement process that
identifies unique and important samples overlooked by existing CIL and enforces
the model to learn from them. Through this process, CILIATE improves the
fairness of CIL by 17.03%, 22.46%, and 31.79% compared to state-of-the-art
methods, iCaRL, BiC, and WA, respectively, based on our evaluation on three
popular datasets and widely used ResNet models.
