  One-to-one (o2o) label assignment plays a key role for transformer based
end-to-end detection, and it has been recently introduced in fully
convolutional detectors for end-to-end dense detection. However, o2o can
degrade the feature learning efficiency due to the limited number of positive
samples. Though extra positive samples are introduced to mitigate this issue in
recent DETRs, the computation of self- and cross- attentions in the decoder
limits its practical application to dense and fully convolutional detectors. In
this work, we propose a simple yet effective one-to-few (o2f) label assignment
strategy for end-to-end dense detection. Apart from defining one positive and
many negative anchors for each object, we define several soft anchors, which
serve as positive and negative samples simultaneously. The positive and
negative weights of these soft anchors are dynamically adjusted during training
so that they can contribute more to ``representation learning'' in the early
training stage, and contribute more to ``duplicated prediction removal'' in the
later stage. The detector trained in this way can not only learn a strong
feature representation but also perform end-to-end dense detection. Experiments
on COCO and CrowdHuman datasets demonstrate the effectiveness of the o2f
scheme. Code is available at https://github.com/strongwolf/o2f.
