  Regularization-based methods are beneficial to alleviate the catastrophic
forgetting problem in class-incremental learning. With the absence of old task
images, they often assume that old knowledge is well preserved if the
classifier produces similar output on new images. In this paper, we find that
their effectiveness largely depends on the nature of old classes: they work
well on classes that are easily distinguishable between each other but may fail
on more fine-grained ones, e.g., boy and girl. In spirit, such methods project
new data onto the feature space spanned by the weight vectors in the fully
connected layer, corresponding to old classes. The resulting projections would
be similar on fine-grained old classes, and as a consequence the new classifier
will gradually lose the discriminative ability on these classes. To address
this issue, we propose a memory-free generative replay strategy to preserve the
fine-grained old classes characteristics by generating representative old
images directly from the old classifier and combined with new data for new
classifier training. To solve the homogenization problem of the generated
samples, we also propose a diversity loss that maximizes Kullback Leibler (KL)
divergence between generated samples. Our method is best complemented by prior
regularization-based methods proved to be effective for easily distinguishable
old classes. We validate the above design and insights on CUB-200-2011,
Caltech-101, CIFAR-100 and Tiny ImageNet and show that our strategy outperforms
existing memory-free methods with a clear margin. Code is available at
https://github.com/xmengxin/MFGR
