  Visual localization is a core component in many applications, including
augmented reality (AR). Localization algorithms compute the camera pose of a
query image w.r.t. a scene representation, which is typically built from
images. This often requires capturing and storing large amounts of data,
followed by running Structure-from-Motion (SfM) algorithms. An interesting, and
underexplored, source of data for building scene representations are 3D models
that are readily available on the Internet, e.g., hand-drawn CAD models, 3D
models generated from building footprints, or from aerial images. These models
allow to perform visual localization right away without the time-consuming
scene capturing and model building steps. Yet, it also comes with challenges as
the available 3D models are often imperfect reflections of reality. E.g., the
models might only have generic or no textures at all, might only provide a
simple approximation of the scene geometry, or might be stretched. This paper
studies how the imperfections of these models affect localization accuracy. We
create a new benchmark for this task and provide a detailed experimental
evaluation based on multiple 3D models per scene. We show that 3D models from
the Internet show promise as an easy-to-obtain scene representation. At the
same time, there is significant room for improvement for visual localization
pipelines. To foster research on this interesting and challenging task, we
release our benchmark at v-pnk.github.io/cadloc.
