  In this paper, we address the problem of distillation-based class-incremental
learning with a single head. A central theme of this task is to learn new
classes that arrive in sequential phases over time while keeping the model's
capability of recognizing seen classes with only limited memory for preserving
seen data samples. Many regularization strategies have been proposed to
mitigate the phenomenon of catastrophic forgetting. To understand better the
essence of these regularizations, we introduce a feature-graph preservation
perspective. Insights into their merits and faults motivate our
weighted-Euclidean regularization for old knowledge preservation. We further
propose rectified cosine normalization and show how it can work with binary
cross-entropy to increase class separation for effective learning of new
classes. Experimental results on both CIFAR-100 and ImageNet datasets
demonstrate that our method outperforms the state-of-the-art approaches in
reducing classification error, easing catastrophic forgetting, and encouraging
evenly balanced accuracy over different classes. Our project page is at :
https://github.com/yhchen12101/FGP-ICL.
