  Deep neural network (DNN) suffers from catastrophic forgetting when learning
incrementally, which greatly limits its applications. Although maintaining a
handful of samples (called `exemplars`) of each task could alleviate forgetting
to some extent, existing methods are still limited by the small number of
exemplars since these exemplars are too few to carry enough task-specific
knowledge, and therefore the forgetting remains. To overcome this problem, we
propose to `imagine` diverse counterparts of given exemplars referring to the
abundant semantic-irrelevant information from unlabeled data. Specifically, we
develop a learnable feature generator to diversify exemplars by adaptively
generating diverse counterparts of exemplars based on semantic information from
exemplars and semantically-irrelevant information from unlabeled data. We
introduce semantic contrastive learning to enforce the generated samples to be
semantic consistent with exemplars and perform semanticdecoupling contrastive
learning to encourage diversity of generated samples. The diverse generated
samples could effectively prevent DNN from forgetting when learning new tasks.
Our method does not bring any extra inference cost and outperforms
state-of-the-art methods on two benchmarks CIFAR-100 and ImageNet-Subset by a
clear margin.
