  Reliability of SLAM systems is considered one of the critical requirements in
modern autonomous systems. This directed the efforts to developing many
state-of-the-art systems, creating challenging datasets, and introducing
rigorous metrics to measure SLAM performance. However, the link between
datasets and performance in the robustness/resilience context has rarely been
explored. In order to fill this void, characterization of the operating
conditions of SLAM systems is essential in order to provide an environment for
quantitative measurement of robustness and resilience. In this paper, we argue
that for proper evaluation of SLAM performance, the characterization of SLAM
datasets serves as a critical first step. The study starts by reviewing
previous efforts for quantitative characterization of SLAM datasets. Then, the
problem of perturbation characterization is discussed and the linkage to SLAM
robustness/resilience is established. After that, we propose a novel, generic
and extendable framework for quantitative analysis and comparison of SLAM
datasets. Additionally, a description of different characterization parameters
is provided. Finally, we demonstrate the application of our framework by
presenting the characterization results of three SLAM datasets: KITTI,
EuroC-MAV, and TUM-VI highlighting the level of insights achieved by the
proposed framework.
