  Despite the success of deep learning models on instance segmentation, current
methods still suffer from catastrophic forgetting in continual learning
scenarios. In this paper, our contributions for continual instance segmentation
are threefold. First, we propose the Y-knowledge distillation (Y-KD), a
technique that shares a common feature extractor between the teacher and
student networks. As the teacher is also updated with new data in Y-KD, the
increased plasticity results in new modules that are specialized on new
classes. Second, our Y-KD approach is supported by a dynamic architecture
method that trains task-specific modules with a unique instance segmentation
head, thereby significantly reducing forgetting. Third, we complete our
approach by leveraging checkpoint averaging as a simple method to manually
balance the trade-off between performance on the various sets of classes, thus
increasing control over the model's behavior without any additional cost. These
contributions are united in our model that we name the Dynamic Y-KD network.
  We perform extensive experiments on several single-step and multi-steps
incremental learning scenarios, and we show that our approach outperforms
previous methods both on past and new classes. For instance, compared to recent
work, our method obtains +2.1% mAP on old classes in 15-1, +7.6% mAP on new
classes in 19-1 and reaches 91.5% of the mAP obtained by joint-training on all
classes in 15-5.
