  Running numerous experiments in simulation is a necessary step before
deploying a control system on a real robot. In this paper we introduce a novel
benchmark that is aimed at quantitatively evaluating the quality of
vision-based simultaneous localization and mapping (vSLAM) and map merging
algorithms. The benchmark consists of both a dataset and a set of tools for
automatic evaluation. The dataset is photo-realistic and provides both the
localization and the map ground truth data. This makes it possible to evaluate
not only the localization part of the SLAM pipeline but the mapping part as
well. To compare the vSLAM-built maps and the ground-truth ones we introduce a
novel way to find correspondences between them that takes the SLAM context into
account (as opposed to other approaches like nearest neighbors). The benchmark
is ROS-compatable and is open-sourced to the community.
  The data and the code are available at: \texttt{github.com/CnnDepth/MAOMaps}.
