  Traditional object detectors are ill-equipped for incremental learning.
However, fine-tuning directly on a well-trained detection model with only new
data will lead to catastrophic forgetting. Knowledge distillation is a flexible
way to mitigate catastrophic forgetting. In Incremental Object Detection (IOD),
previous work mainly focuses on distilling for the combination of features and
responses. However, they under-explore the information that contains in
responses. In this paper, we propose a response-based incremental distillation
method, dubbed Elastic Response Distillation (ERD), which focuses on
elastically learning responses from the classification head and the regression
head. Firstly, our method transfers category knowledge while equipping student
detector with the ability to retain localization information during incremental
learning. In addition, we further evaluate the quality of all locations and
provide valuable responses by the Elastic Response Selection (ERS) strategy.
Finally, we elucidate that the knowledge from different responses should be
assigned with different importance during incremental distillation. Extensive
experiments conducted on MS COCO demonstrate our method achieves
state-of-the-art result, which substantially narrows the performance gap
towards full training.
