  Continual learning poses a fundamental challenge for modern machine learning
systems, requiring models to adapt to new tasks while retaining knowledge from
previous ones. Addressing this challenge necessitates the development of
efficient algorithms capable of learning from data streams and accumulating
knowledge over time. This paper proposes a novel approach to continual learning
utilizing the weight consolidation method. Our method, a simple yet powerful
technique, enhances robustness against catastrophic forgetting by interpolating
between old and new model weights after each novel task, effectively merging
two models to facilitate exploration of local minima emerging after arrival of
new concepts. Moreover, we demonstrate that our approach can complement
existing rehearsal-based replay approaches, improving their accuracy and
further mitigating the forgetting phenomenon. Additionally, our method provides
an intuitive mechanism for controlling the stability-plasticity trade-off.
Experimental results showcase the significant performance enhancement to
state-of-the-art experience replay algorithms the proposed weight consolidation
approach offers. Our algorithm can be downloaded from
https://github.com/jedrzejkozal/weight-interpolation-cl.
