  Feature pyramid architecture has been broadly adopted in object detection and
segmentation to deal with multi-scale problem. However, in this paper we show
that the capacity of the architecture has not been fully explored due to the
inadequate utilization of the supervision information. Such insufficient
utilization is caused by the supervision signal degradation in back
propagation. Thus inspired, we propose a dually supervised method, named dually
supervised FPN (DSFPN), to enhance the supervision signal when training the
feature pyramid network (FPN). In particular, DSFPN is constructed by attaching
extra prediction (i.e., detection or segmentation) heads to the bottom-up
subnet of FPN. Hence, the features can be optimized by the additional heads
before being forwarded to subsequent networks. Further, the auxiliary heads can
serve as a regularization term to facilitate the model training. In addition,
to strengthen the capability of the detection heads in DSFPN for handling two
inhomogeneous tasks, i.e., classification and regression, the originally shared
hidden feature space is separated by decoupling classification and regression
subnets. To demonstrate the generalizability, effectiveness, and efficiency of
the proposed method, DSFPN is integrated into four representative detectors
(Faster RCNN, Mask RCNN, Cascade RCNN, and Cascade Mask RCNN) and assessed on
the MS COCO dataset. Promising precision improvement, state-of-the-art
performance, and negligible additional computational cost are demonstrated
through extensive experiments. Code will be provided.
