  Online continual learning (OCL) involves deep neural networks retaining
knowledge from old data while adapting to new data, which is accessible only
once. A critical challenge in OCL is catastrophic forgetting, reflected in
reduced model performance on old data. Existing replay-based methods mitigate
forgetting by replaying buffered samples from old data and learning current
samples of new data. In this work, we dissect existing methods and empirically
discover that learning and replaying in the same feature space is not conducive
to addressing the forgetting issue. Since the learned features associated with
old data are readily changed by the features related to new data due to data
imbalance, leading to the forgetting problem. Based on this observation, we
intuitively explore learning and replaying in different feature spaces.
Learning in a feature subspace is sufficient to capture novel knowledge from
new data while replaying in a larger feature space provides more feature space
to maintain historical knowledge from old data. To this end, we propose a novel
OCL approach called experience replay with feature subspace learning (ER-FSL).
Firstly, ER-FSL divides the entire feature space into multiple subspaces, with
each subspace used to learn current samples. Moreover, it introduces a subspace
reuse mechanism to address situations where no blank subspaces exist. Secondly,
ER-FSL replays previous samples using an accumulated space comprising all
learned subspaces. Extensive experiments on three datasets demonstrate the
superiority of ER-FSL over various state-of-the-art methods.
