  In continual learning, model needs to continually learn a feature extractor
and classifier on a sequence of tasks. This paper focuses on how to learn a
classifier based on a pretrained feature extractor under continual learning
setting. We present an probabilistic analysis that the failure of vanilla
experience replay (ER) comes from unnecessary re-learning of previous tasks and
incompetence to distinguish current task from the previous ones, which is the
cause of knowledge degradation and prediction bias. To overcome these
weaknesses, we propose a novel replay strategy task-aware experience replay. It
rebalances the replay loss and detaches classifier weight for the old tasks
from the update process, by which the previous knowledge is kept intact and the
overfitting on episodic memory is alleviated. Experimental results show our
method outperforms current state-of-the-art methods.
