  Continual learning (CL) learns a sequence of tasks incrementally. There are
two popular CL settings, class incremental learning (CIL) and task incremental
learning (TIL). A major challenge of CL is catastrophic forgetting (CF). While
a number of techniques are already available to effectively overcome CF for
TIL, CIL remains to be highly challenging. So far, little theoretical study has
been done to provide a principled guidance on how to solve the CIL problem.
This paper performs such a study. It first shows that probabilistically, the
CIL problem can be decomposed into two sub-problems: Within-task Prediction
(WP) and Task-id Prediction (TP). It further proves that TP is correlated with
out-of-distribution (OOD) detection, which connects CIL and OOD detection. The
key conclusion of this study is that regardless of whether WP and TP or OOD
detection are defined explicitly or implicitly by a CIL algorithm, good WP and
good TP or OOD detection are necessary and sufficient for good CIL
performances. Additionally, TIL is simply WP. Based on the theoretical result,
new CIL methods are also designed, which outperform strong baselines in both
CIL and TIL settings by a large margin.
