  Previous object detectors make predictions based on dense grid points or
numerous preset anchors. Most of these detectors are trained with one-to-many
label assignment strategies. On the contrary, recent query-based object
detectors depend on a sparse set of learnable queries and a series of decoder
layers. The one-to-one label assignment is independently applied on each layer
for the deep supervision during training. Despite the great success of
query-based object detection, however, this one-to-one label assignment
strategy demands the detectors to have strong fine-grained discrimination and
modeling capacity. To solve the above problems, in this paper, we propose a new
query-based object detector with cross-stage interaction, coined as
StageInteractor. During the forward propagation, we come up with an efficient
way to improve this modeling ability by reusing dynamic operators with
lightweight adapters. As for the label assignment, a cross-stage label assigner
is applied subsequent to the one-to-one label assignment. With this assigner,
the training target class labels are gathered across stages and then
reallocated to proper predictions at each decoder layer. On MS COCO benchmark,
our model improves the baseline by 2.2 AP, and achieves 44.8 AP with ResNet-50
as backbone, 100 queries and 12 training epochs. With longer training time and
300 queries, StageInteractor achieves 51.1 AP and 52.2 AP with ResNeXt-101-DCN
and Swin-S, respectively.
