  Catastrophic forgetting of previously learned knowledge while learning new
tasks is a widely observed limitation of contemporary neural networks. Although
many continual learning methods are proposed to mitigate this drawback, the
main question remains unanswered: what is the root cause of catastrophic
forgetting? In this work, we aim at answering this question by posing and
validating a set of research hypotheses related to the specificity of
representations built internally by neural models. More specifically, we design
a set of empirical evaluations that compare the robustness of representations
in discriminative and generative models against catastrophic forgetting. We
observe that representations learned by discriminative models are more prone to
catastrophic forgetting than their generative counterparts, which sheds new
light on the advantages of developing generative models for continual learning.
Finally, our work opens new research pathways and possibilities to adopt
generative models in continual learning beyond mere replay mechanisms.
