  Despite their effectiveness in a wide range of tasks, deep architectures
suffer from some important limitations. In particular, they are vulnerable to
catastrophic forgetting, i.e. they perform poorly when they are required to
update their model as new classes are available but the original training set
is not retained. This paper addresses this problem in the context of semantic
segmentation. Current strategies fail on this task because they do not consider
a peculiar aspect of semantic segmentation: since each training step provides
annotation only for a subset of all possible classes, pixels of the background
class (i.e. pixels that do not belong to any other classes) exhibit a semantic
distribution shift. In this work we revisit classical incremental learning
methods, proposing a new distillation-based framework which explicitly accounts
for this shift. Furthermore, we introduce a novel strategy to initialize
classifier's parameters, thus preventing biased predictions toward the
background class. We demonstrate the effectiveness of our approach with an
extensive evaluation on the Pascal-VOC 2012 and ADE20K datasets, significantly
outperforming state of the art incremental learning methods.
