  Benchmarking Simultaneous Localization and Mapping (SLAM) algorithms is
important to scientists and users of robotic systems alike. But through their
many configuration options in hardware and software, SLAM systems feature a
vast parameter space that scientists up to now were not able to explore. The
proposed SLAM Hive Benchmarking Suite is able to analyze SLAM algorithms in
1000's of mapping runs, through its utilization of container technology and
deployment in a cluster. This paper presents the architecture and open source
implementation of SLAM Hive and compares it to existing efforts on SLAM
evaluation. Furthermore, we highlight the function of SLAM Hive by exploring
some open source algorithms on public datasets in terms of accuracy. We compare
the algorithms against each other and evaluate how parameters effect not only
accuracy but also CPU and memory usage. Through this we show that SLAM Hive can
become an essential tool for proper comparisons and evaluations of SLAM
algorithms and thus drive the scientific development in the research on SLAM.
