  Feature pyramid network (FPN) is a critical component in modern object
detection frameworks. The performance gain in most of the existing FPN variants
is mainly attributed to the increase of computational burden. An attempt to
enhance the FPN is enriching the spatial information by expanding the receptive
fields, which is promising to largely improve the detection accuracy. In this
paper, we first investigate how expanding the receptive fields affect the
accuracy and computational costs of FPN. We explore a baseline model called
inception FPN in which each lateral connection contains convolution filters
with different kernel sizes. Moreover, we point out that not all objects need
such a complicated calculation and propose a new dynamic FPN (DyFPN). The
output features of DyFPN will be calculated by using the adaptively selected
branch according to a dynamic gating operation. Therefore, the proposed method
can provide a more efficient dynamic inference for achieving a better trade-off
between accuracy and computational cost. Extensive experiments conducted on
MS-COCO benchmark demonstrate that the proposed DyFPN significantly improves
performance with the optimal allocation of computation resources. For instance,
replacing inception FPN with DyFPN reduces about 40% of its FLOPs while
maintaining similar high performance.
