  Feature Pyramid Network (FPN) has been an essential module for object
detection models to consider various scales of an object. However, average
precision (AP) on small objects is relatively lower than AP on medium and large
objects. The reason is why the deeper layer of CNN causes information loss as
feature extraction level. We propose a new scale sequence (S^2) feature
extraction of FPN to strengthen feature information of small objects. We
consider FPN structure as scale-space and extract scale sequence (S^2) feature
by 3D convolution on the level axis of FPN. It is basically scale invariant
feature and is built on high-resolution pyramid feature map for small objects.
Furthermore, the proposed S^2 feature can be extended to most object detection
models based on FPN. We demonstrate the proposed S2 feature can improve the
performance of both one-stage and two-stage detectors on MS COCO dataset. Based
on the proposed S2 feature, we achieve upto 1.3% and 1.1% of AP improvement for
YOLOv4-P5 and YOLOv4-P6, respectively. For Faster RCNN and Mask R-CNN, we
observe upto 2.0% and 1.6% of AP improvement with the suggested S^2 feature,
respectively.
