  Continual Learning (CL) aims at incrementally learning new tasks without
forgetting the knowledge acquired from old ones. Experience Replay (ER) is a
simple and effective rehearsal-based strategy, which optimizes the model with
current training data and a subset of old samples stored in a memory buffer. To
further reduce forgetting, recent approaches extend ER with various techniques,
such as model regularization and memory sampling. However, the prediction
consistency between the new model and the old one on current training data has
been seldom explored, resulting in less knowledge preserved when few previous
samples are available. To address this issue, we propose a CL method with
Strong Experience Replay (SER), which additionally utilizes future experiences
mimicked on the current training data, besides distilling past experience from
the memory buffer. In our method, the updated model will produce approximate
outputs as its original ones, which can effectively preserve the acquired
knowledge. Experimental results on multiple image classification datasets show
that our SER method surpasses the state-of-the-art methods by a noticeable
margin.
