  Class-incremental learning (CIL) is a particularly challenging variant of
continual learning, where the goal is to learn to discriminate between all
classes presented in an incremental fashion. Existing approaches often suffer
from excessive forgetting and imbalance of the scores assigned to classes that
have not been seen together during training. In this study, we introduce a
novel approach, Prediction Error-based Classification (PEC), which differs from
traditional discriminative and generative classification paradigms. PEC
computes a class score by measuring the prediction error of a model trained to
replicate the outputs of a frozen random neural network on data from that
class. The method can be interpreted as approximating a classification rule
based on Gaussian Process posterior variance. PEC offers several practical
advantages, including sample efficiency, ease of tuning, and effectiveness even
when data are presented one class at a time. Our empirical results show that
PEC performs strongly in single-pass-through-data CIL, outperforming other
rehearsal-free baselines in all cases and rehearsal-based methods with moderate
replay buffer size in most cases across multiple benchmarks.
