  Time-dependent data-generating distributions have proven to be difficult for
gradient-based training of neural networks, as the greedy updates result in
catastrophic forgetting of previously learned knowledge. Despite the progress
in the field of continual learning to overcome this forgetting, we show that a
set of common state-of-the-art methods still suffers from substantial
forgetting upon starting to learn new tasks, except that this forgetting is
temporary and followed by a phase of performance recovery. We refer to this
intriguing but potentially problematic phenomenon as the stability gap. The
stability gap had likely remained under the radar due to standard practice in
the field of evaluating continual learning models only after each task.
Instead, we establish a framework for continual evaluation that uses
per-iteration evaluation and we define a new set of metrics to quantify
worst-case performance. Empirically we show that experience replay,
constraint-based replay, knowledge-distillation, and parameter regularization
methods are all prone to the stability gap; and that the stability gap can be
observed in class-, task-, and domain-incremental learning benchmarks.
Additionally, a controlled experiment shows that the stability gap increases
when tasks are more dissimilar. Finally, by disentangling gradients into
plasticity and stability components, we propose a conceptual explanation for
the stability gap.
