  Currently, reliable and accurate ship detection in optical remote sensing
images is still challenging. Even the state-of-the-art convolutional neural
network (CNN) based methods cannot obtain very satisfactory results. To more
accurately locate the ships in diverse orientations, some recent methods
conduct the detection via the rotated bounding box. However, it further
increases the difficulty of detection, because an additional variable of ship
orientation must be accurately predicted in the algorithm. In this paper, a
novel CNN-based ship detection method is proposed, by overcoming some common
deficiencies of current CNN-based methods in ship detection. Specifically, to
generate rotated region proposals, current methods have to predefine
multi-oriented anchors, and predict all unknown variables together in one
regression process, limiting the quality of overall prediction. By contrast, we
are able to predict the orientation and other variables independently, and yet
more effectively, with a novel dual-branch regression network, based on the
observation that the ship targets are nearly rotation-invariant in remote
sensing images. Next, a shape-adaptive pooling method is proposed, to overcome
the limitation of typical regular ROI-pooling in extracting the features of the
ships with various aspect ratios. Furthermore, we propose to incorporate
multilevel features via the spatially-variant adaptive pooling. This novel
approach, called multilevel adaptive pooling, leads to a compact feature
representation more qualified for the simultaneous ship classification and
localization. Finally, detailed ablation study performed on the proposed
approaches is provided, along with some useful insights. Experimental results
demonstrate the great superiority of the proposed method in ship detection.
