  Most continual learning (CL) algorithms have focused on tackling the
stability-plasticity dilemma, that is, the challenge of preventing the
forgetting of previous tasks while learning new ones. However, they have
overlooked the impact of the knowledge transfer when the dataset in a certain
task is biased - namely, when some unintended spurious correlations of the
tasks are learned from the biased dataset. In that case, how would they affect
learning future tasks or the knowledge already learned from the past tasks? In
this work, we carefully design systematic experiments using one synthetic and
two real-world datasets to answer the question from our empirical findings.
Specifically, we first show through two-task CL experiments that standard CL
methods, which are unaware of dataset bias, can transfer biases from one task
to another, both forward and backward, and this transfer is exacerbated
depending on whether the CL methods focus on the stability or the plasticity.
We then present that the bias transfer also exists and even accumulate in
longer sequences of tasks. Finally, we propose a simple, yet strong plug-in
method for debiasing-aware continual learning, dubbed as Group-class Balanced
Greedy Sampling (BGS). As a result, we show that our BGS can always reduce the
bias of a CL model, with a slight loss of CL performance at most.
