  Recently, anchor-free detection methods have been through great progress. The
major two families, anchor-point detection and key-point detection, are at
opposite edges of the speed-accuracy trade-off, with anchor-point detectors
having the speed advantage. In this work, we boost the performance of the
anchor-point detector over the key-point counterparts while maintaining the
speed advantage. To achieve this, we formulate the detection problem from the
anchor point's perspective and identify ineffective training as the main
problem. Our key insight is that anchor points should be optimized jointly as a
group both within and across feature pyramid levels. We propose a simple yet
effective training strategy with soft-weighted anchor points and soft-selected
pyramid levels to address the false attention issue within each pyramid level
and the feature selection issue across all the pyramid levels, respectively. To
evaluate the effectiveness, we train a single-stage anchor-free detector called
Soft Anchor-Point Detector (SAPD). Experiments show that our concise SAPD
pushes the envelope of speed/accuracy trade-off to a new level, outperforming
recent state-of-the-art anchor-free and anchor-based detectors. Without bells
and whistles, our best model can achieve a single-model single-scale AP of
47.4% on COCO.
