  Object detection is a challenging task in remote sensing because objects only
occupy a few pixels in the images, and the models are required to
simultaneously learn object locations and detection. Even though the
established approaches well perform for the objects of regular sizes, they
achieve weak performance when analyzing small ones or getting stuck in the
local minima (e.g. false object parts). Two possible issues stand in their way.
First, the existing methods struggle to perform stably on the detection of
small objects because of the complicated background. Second, most of the
standard methods used hand-crafted features, and do not work well on the
detection of objects parts of which are missing. We here address the above
issues and propose a new architecture with a multiple patch feature pyramid
network (MPFP-Net). Different from the current models that during training only
pursue the most discriminative patches, in MPFPNet the patches are divided into
class-affiliated subsets, in which the patches are related and based on the
primary loss function, a sequence of smooth loss functions are determined for
the subsets to improve the model for collecting small object parts. To enhance
the feature representation for patch selection, we introduce an effective
method to regularize the residual values and make the fusion transition layers
strictly norm-preserving. The network contains bottom-up and crosswise
connections to fuse the features of different scales to achieve better
accuracy, compared to several state-of-the-art object detection models. Also,
the developed architecture is more efficient than the baselines.
