  We propose a simple yet effective proposal-based object detector, aiming at
detecting highly-overlapped instances in crowded scenes. The key of our
approach is to let each proposal predict a set of correlated instances rather
than a single one in previous proposal-based frameworks. Equipped with new
techniques such as EMD Loss and Set NMS, our detector can effectively handle
the difficulty of detecting highly overlapped objects. On a FPN-Res50 baseline,
our detector can obtain 4.9\% AP gains on challenging CrowdHuman dataset and
1.0\% $\text{MR}^{-2}$ improvements on CityPersons dataset, without bells and
whistles. Moreover, on less crowed datasets like COCO, our approach can still
achieve moderate improvement, suggesting the proposed method is robust to
crowdedness. Code and pre-trained models will be released at
https://github.com/megvii-model/CrowdDetection.
