  Lifelong learning capabilities are crucial for artificial autonomous agents
operating on real-world data, which is typically non-stationary and temporally
correlated. In this work, we demonstrate that dynamically grown networks
outperform static networks in incremental learning scenarios, even when bounded
by the same amount of memory in both cases. Learning is unsupervised in our
models, a condition that additionally makes training more challenging whilst
increasing the realism of the study, since humans are able to learn without
dense manual annotation. Our results on artificial neural networks reinforce
that structural plasticity constitutes effective prevention against
catastrophic forgetting in non-stationary environments, as well as empirically
supporting the importance of neurogenesis in the mammalian brain.
