  Street-to-aerial image geo-localization, which matches a query street-view
image to the GPS-tagged aerial images in a reference set, has attracted
increasing attention recently. In this paper, we revisit this problem and point
out the ignored issue about image alignment information. We show that the
performance of a simple Siamese network is highly dependent on the alignment
setting and the comparison of previous works can be unfair if they have
different assumptions. Instead of focusing on the feature extraction under the
alignment assumption, we show that improvements in metric learning techniques
significantly boost the performance regardless of the alignment. Without
leveraging the alignment information, our pipeline outperforms previous works
on both panorama and cropped datasets. Furthermore, we conduct visualization to
help understand the learned model and the effect of alignment information using
Grad-CAM. With our discovery on the approximate rotation-invariant activation
maps, we propose a novel method to estimate the orientation/alignment between a
pair of cross-view images with unknown alignment information. It achieves
state-of-the-art results on the CVUSA dataset.
