  With the recent development of autonomous vehicle technology, there have been
active efforts on the deployment of this technology at different scales that
include urban and highway driving. While many of the prototypes showcased have
been shown to operate under specific cases, little effort has been made to
better understand their shortcomings and generalizability to new areas.
Distance, uptime and number of manual disengagements performed during
autonomous driving provide a high-level idea on the performance of an
autonomous system but without proper data normalization, testing location
information, and the number of vehicles involved in testing, the disengagement
reports alone do not fully encompass system performance and robustness. Thus,
in this study a complete set of metrics are applied for benchmarking autonomous
vehicle systems in a variety of scenarios that can be extended for comparison
with human drivers and other autonomous vehicle systems. These metrics have
been used to benchmark UC San Diego's autonomous vehicle platforms during early
deployments for micro-transit and autonomous mail delivery applications.
