  Large pre-trained language models help to achieve state of the art on a
variety of natural language processing (NLP) tasks, nevertheless, they still
suffer from forgetting when incrementally learning a sequence of tasks. To
alleviate this problem, recent works enhance existing models by sparse
experience replay and local adaption, which yield satisfactory performance.
However, in this paper we find that pre-trained language models like BERT have
a potential ability to learn sequentially, even without any sparse memory
replay. To verify the ability of BERT to maintain old knowledge, we adopt and
re-finetune single-layer probe networks with the parameters of BERT fixed. We
investigate the models on two types of NLP tasks, text classification and
extractive question answering. Our experiments reveal that BERT can actually
generate high quality representations for previously learned tasks in a long
term, under extremely sparse replay or even no replay. We further introduce a
series of novel methods to interpret the mechanism of forgetting and how memory
rehearsal plays a significant role in task incremental learning, which bridges
the gap between our new discovery and previous studies about catastrophic
forgetting.
