  Successful continual learning of new knowledge would enable intelligent
systems to recognize more and more classes of objects. However, current
intelligent systems often fail to correctly recognize previously learned
classes of objects when updated to learn new classes. It is widely believed
that such downgraded performance is solely due to the catastrophic forgetting
of previously learned knowledge. In this study, we argue that the class
confusion phenomena may also play a role in downgrading the classification
performance during continual learning, i.e., the high similarity between new
classes and any previously learned classes would also cause the classifier to
make mistakes in recognizing these old classes, even if the knowledge of these
old classes is not forgotten. To alleviate the class confusion issue, we
propose a discriminative distillation strategy to help the classify well learn
the discriminative features between confusing classes during continual
learning. Experiments on multiple natural image classification tasks support
that the proposed distillation strategy, when combined with existing methods,
is effective in further improving continual learning.
