  The recent surge of pervasive devices that generate dynamic data streams has
underscored the necessity for learning systems to adapt continually to data
distributional shifts. To tackle this challenge, the research community has put
forth a spectrum of methodologies, including the demanding pursuit of
class-incremental learning without replay data. In this study, we present MIND,
a parameter isolation method that aims to significantly enhance the performance
of replay-free solutions and achieve state-of-the-art results on several widely
studied datasets. Our approach introduces two main contributions: two
alternative distillation procedures that significantly improve the efficiency
of MIND increasing the accumulated knowledge of each sub-network, and the
optimization of the BachNorm layers across tasks inside the sub-networks.
Overall, MIND outperforms all the state-of-the-art methods for rehearsal-free
Class-Incremental learning (with an increment in classification accuracy of
approx. +6% on CIFAR-100/10 and +10% on TinyImageNet/10) reaching up to approx.
+40% accuracy in Domain-Incremental scenarios. Moreover, we ablated each
contribution to demonstrate its impact on performance improvement. Our results
showcase the superior performance of MIND indicating its potential for
addressing the challenges posed by Class-incremental and Domain-Incremental
learning in resource-constrained environments.
