  Sharpening deep learning models by training them with examples close to the
decision boundary is a well-known best practice. Nonetheless, these models are
still error-prone in producing predictions. In practice, the inference of the
deep learning models in many application systems is guarded by a rejector, such
as a confidence-based rejector, to filter out samples with insufficient
prediction confidence. Such confidence-based rejectors cannot effectively guard
against failing samples with high confidence. Existing test case prioritization
techniques effectively distinguish confusing samples from confident samples to
identify failing samples among the confusing ones, yet prioritizing the failing
ones high among many confident ones is challenging. In this paper, we propose
$A^3$Rank, a novel test case prioritization technique with augmentation
alignment analysis, to address this problem. $A^3$Rank generates augmented
versions of each test case and assesses the extent of the prediction result for
the test case misaligned with these of the augmented versions and vice versa.
Our experiment shows that $A^3$Rank can effectively rank failing samples
escaping from the checking of confidence-based rejectors, which significantly
outperforms the peer techniques by 163.63\% in the detection ratio of
top-ranked samples. We also provide a framework to construct a detector devoted
to augmenting these rejectors to defend these failing samples, and our detector
can achieve a significantly higher defense success rate.
