  This work introduces a simulator-based benchmark for visual localization in
the autonomous navigation context. The dynamic benchmark enables investigation
of how variables such as the time of day, weather, and camera perspective
affect the navigation performance of autonomous agents that utilize visual
localization for closed-loop control. The experimental part of the paper
studies the effects of four such variables by evaluating state-of-the-art
visual localization methods as part of the motion planning module of an
autonomous navigation stack. The results show major variation in the
suitability of the different methods for vision-based navigation. To the
authors' best knowledge, the proposed benchmark is the first to study modern
visual localization methods as part of a complete navigation stack. We make the
benchmark available at https://github.com/lasuomela/carla_vloc_benchmark.
