  We propose a novel learning-based formulation for visual localization of
vehicles that can operate in real-time in city-scale environments. Visual
localization algorithms determine the position and orientation from which an
image has been captured, using a set of geo-referenced images or a 3D scene
representation. Our new localization paradigm, named Implicit Pose Encoding
(ImPosing), embeds images and camera poses into a common latent representation
with 2 separate neural networks, such that we can compute a similarity score
for each image-pose pair. By evaluating candidates through the latent space in
a hierarchical manner, the camera position and orientation are not directly
regressed but incrementally refined. Very large environments force competitors
to store gigabytes of map data, whereas our method is very compact
independently of the reference database size. In this paper, we describe how to
effectively optimize our learned modules, how to combine them to achieve
real-time localization, and demonstrate results on diverse large scale
scenarios that significantly outperform prior work in accuracy and
computational efficiency.
