  Robustness and safety are crucial properties for the real-world application
of autonomous vehicles. One of the most critical components of any autonomous
system is localisation. During the last 20 years there has been significant
progress in this area with the introduction of very efficient algorithms for
mapping, localisation and SLAM. Many of these algorithms present impressive
demonstrations for a particular domain, but fail to operate reliably with
changes to the operating environment. The aspect of robustness has not received
enough attention and localisation systems for self-driving vehicle applications
are seldom evaluated for their robustness. In this paper we propose novel
metrics to effectively quantify localisation robustness with or without an
accurate ground truth. The experimental results present a comprehensive
analysis of the application of these metrics against a number of well known
localisation strategies.
