  We propose three novel metrics for evaluating the accuracy of a set of
estimated camera poses given the ground truth: Translation Alignment Score
(TAS), Rotation Alignment Score (RAS), and Pose Alignment Score (PAS). The TAS
evaluates the translation accuracy independently of the rotations, and the RAS
evaluates the rotation accuracy independently of the translations. The PAS is
the average of the two scores, evaluating the combined accuracy of both
translations and rotations. The TAS is computed in four steps: (1) Find the
upper quartile of the closest-pair-distances, $d$. (2) Align the estimated
trajectory to the ground truth using a robust registration method. (3) Collect
all distance errors and obtain the cumulative frequencies for multiple
thresholds ranging from $0.01d$ to $d$ with a resolution $0.01d$. (4) Add up
these cumulative frequencies and normalize them such that the theoretical
maximum is 1. The TAS has practical advantages over the existing metrics in
that (1) it is robust to outliers and collinear motion, and (2) there is no
need to adjust parameters on different datasets. The RAS is computed in a
similar manner to the TAS and is also shown to be more robust against outliers
than the existing rotation metrics. We verify our claims through extensive
simulations and provide in-depth discussion of the strengths and weaknesses of
the proposed metrics.
