  In Class-Incremental Learning (CIL) an image classification system is exposed
to new classes in each learning session and must be updated incrementally.
Methods approaching this problem have updated both the classification head and
the feature extractor body at each session of CIL. In this work, we develop a
baseline method, First Session Adaptation (FSA), that sheds light on the
efficacy of existing CIL approaches and allows us to assess the relative
performance contributions from head and body adaption. FSA adapts a pre-trained
neural network body only on the first learning session and fixes it thereafter;
a head based on linear discriminant analysis (LDA), is then placed on top of
the adapted body, allowing exact updates through CIL. FSA is replay-free
i.e.~it does not memorize examples from previous sessions of continual
learning. To empirically motivate FSA, we first consider a diverse selection of
22 image-classification datasets, evaluating different heads and body
adaptation techniques in high/low-shot offline settings. We find that the LDA
head performs well and supports CIL out-of-the-box. We also find that
Featurewise Layer Modulation (FiLM) adapters are highly effective in the
few-shot setting, and full-body adaption in the high-shot setting. Second, we
empirically investigate various CIL settings including high-shot CIL and
few-shot CIL, including settings that have previously been used in the
literature. We show that FSA significantly improves over the state-of-the-art
in 15 of the 16 settings considered. FSA with FiLM adapters is especially
performant in the few-shot setting. These results indicate that current
approaches to continuous body adaptation are not working as expected. Finally,
we propose a measure that can be applied to a set of unlabelled inputs which is
predictive of the benefits of body adaptation.
