  We introduce BenchBot, a novel software suite for benchmarking the
performance of robotics research across both photorealistic 3D simulations and
real robot platforms. BenchBot provides a simple interface to the sensorimotor
capabilities of a robot when solving robotics research problems; an interface
that is consistent regardless of whether the target platform is simulated or a
real robot. In this paper we outline the BenchBot system architecture, and
explore the parallels between its user-centric design and an ideal research
development process devoid of tangential robot engineering challenges. The
paper describes the research benefits of using the BenchBot system, including:
enhanced capacity to focus solely on research problems, direct quantitative
feedback to inform research development, tools for deriving comprehensive
performance characteristics, and submission formats which promote sharability
and repeatability of research outcomes. BenchBot is publicly available
(http://benchbot.org), and we encourage its use in the research community for
comprehensively evaluating the simulated and real world performance of novel
robotic algorithms.
