  Catastrophic forgetting (CF) occurs when a neural network loses the
information previously learned while training on a set of samples from a
different distribution, i.e., a new task. Existing approaches have achieved
remarkable results in mitigating CF, especially in a scenario called task
incremental learning. However, this scenario is not realistic, and limited work
has been done to achieve good results on more realistic scenarios. In this
paper, we propose a novel regularization method called Centroids Matching,
that, inspired by meta-learning approaches, fights CF by operating in the
feature space produced by the neural network, achieving good results while
requiring a small memory footprint. Specifically, the approach classifies the
samples directly using the feature vectors produced by the neural network, by
matching those vectors with the centroids representing the classes from the
current task, or all the tasks up to that point. Centroids Matching is faster
than competing baselines, and it can be exploited to efficiently mitigate CF,
by preserving the distances between the embedding space produced by the model
when past tasks were over, and the one currently produced, leading to a method
that achieves high accuracy on all the tasks, without using an external memory
when operating on easy scenarios, or using a small one for more realistic ones.
Extensive experiments demonstrate that Centroids Matching achieves accuracy
gains on multiple datasets and scenarios.
