  Lifelong learning has attracted much attention, but existing works still
struggle to fight catastrophic forgetting and accumulate knowledge over long
stretches of incremental learning. In this work, we propose PODNet, a model
inspired by representation learning. By carefully balancing the compromise
between remembering the old classes and learning new ones, PODNet fights
catastrophic forgetting, even over very long runs of small incremental tasks
--a setting so far unexplored by current works. PODNet innovates on existing
art with an efficient spatial-based distillation-loss applied throughout the
model and a representation comprising multiple proxy vectors for each class. We
validate those innovations thoroughly, comparing PODNet with three
state-of-the-art models on three datasets: CIFAR100, ImageNet100, and
ImageNet1000. Our results showcase a significant advantage of PODNet over
existing art, with accuracy gains of 12.10, 6.51, and 2.85 percentage points,
respectively. Code is available at
https://github.com/arthurdouillard/incremental_learning.pytorch
