  In the SSLAD-Track 3B challenge on continual learning, we propose the method
of COntinual Learning with Transformer (COLT). We find that transformers suffer
less from catastrophic forgetting compared to convolutional neural network. The
major principle of our method is to equip the transformer based feature
extractor with old knowledge distillation and head expanding strategies to
compete catastrophic forgetting. In this report, we first introduce the overall
framework of continual learning for object detection. Then, we analyse the key
elements' effect on withstanding catastrophic forgetting in our solution. Our
method achieves 70.78 mAP on the SSLAD-Track 3B challenge test set.
