  Modern object detection methods based on convolutional neural network suffer
from severe catastrophic forgetting in learning new classes without original
data. Due to time consumption, storage burden and privacy of old data, it is
inadvisable to train the model from scratch with both old and new data when new
object classes emerge after the model trained. In this paper, we propose a
novel incremental object detector based on Faster R-CNN to continuously learn
from new object classes without using old data. It is a triple network where an
old model and a residual model as assistants for helping the incremental model
learning on new classes without forgetting the previous learned knowledge. To
better maintain the discrimination of features between old and new classes, the
residual model is jointly trained on new classes in the incremental learning
procedure. In addition, a corresponding distillation scheme is designed to
guide the training process, which consists of a two-level residual distillation
loss and a joint classification distillation loss. Extensive experiments on
VOC2007 and COCO are conducted, and the results demonstrate that the proposed
method can effectively learn to incrementally detect objects of new classes,
and the problem of catastrophic forgetting is mitigated in this context.
