  Most existing approaches for visual localization either need a detailed 3D
model of the environment or, in the case of learning-based methods, must be
retrained for each new scene. This can either be very expensive or simply
impossible for large, unknown environments, for example in search-and-rescue
scenarios. Although there are learning-based approaches that operate
scene-agnostically, the generalization capability of these methods is still
outperformed by classical approaches. In this paper, we present an approach
that can generalize to new scenes by applying specific changes to the model
architecture, including an extended regression part, the use of hierarchical
correlation layers, and the exploitation of scale and uncertainty information.
Our approach outperforms the 5-point algorithm using SIFT features on equally
big images and additionally surpasses all previous learning-based approaches
that were trained on different data. It is also superior to most of the
approaches that were specifically trained on the respective scenes. We also
evaluate our approach in a scenario where only very few reference images are
available, showing that under such more realistic conditions our learning-based
approach considerably exceeds both existing learning-based and classical
methods.
