  Object detection aims to localize and classify the objects in a given image,
and these two tasks are sensitive to different object regions. Therefore, some
locations predict high-quality bounding boxes but low classification scores,
and some locations are quite the opposite. A misalignment exists between the
two tasks, and their features are spatially entangled. In order to solve the
misalignment problem, we propose a plug-in Spatial-disentangled and
Task-aligned operator (SALT). By predicting two task-aware point sets that are
located in each task's sensitive regions, SALT can reassign features from those
regions and align them to the corresponding anchor point. Therefore, features
for the two tasks are spatially aligned and disentangled. To minimize the
difference between the two regression stages, we propose a Self-distillation
regression (SDR) loss that can transfer knowledge from the refined regression
results to the coarse regression results. On the basis of SALT and SDR loss, we
propose SALT-Net, which explicitly exploits task-aligned point-set features for
accurate detection results. Extensive experiments on the MS-COCO dataset show
that our proposed methods can consistently boost different state-of-the-art
dense detectors by $\sim$2 AP. Notably, SALT-Net with Res2Net-101-DCN backbone
achieves 53.8 AP on the MS-COCO test-dev.
