  Visual localization is the problem of estimating the position and orientation
from which a given image (or a sequence of images) is taken in a known scene.
It is an important part of a wide range of computer vision and robotics
applications, from self-driving cars to augmented/virtual reality systems.
Visual localization techniques should work reliably and robustly under a wide
range of conditions, including seasonal, weather, illumination and man-made
changes. Recent benchmarking efforts model this by providing images under
different conditions, and the community has made rapid progress on these
datasets since their inception. However, they are limited to a few geographical
regions and often recorded with a single device. We propose a new benchmark for
visual localization in outdoor scenes, using crowd-sourced data to cover a wide
range of geographical regions and camera devices with a focus on the failure
cases of current algorithms. Experiments with state-of-the-art localization
approaches show that our dataset is very challenging, with all evaluated
methods failing on its hardest parts. As part of the dataset release, we
provide the tooling used to generate it, enabling efficient and effective 2D
correspondence annotation to obtain reference poses.
