  Pedestrian re-identification (ReID) is the task of continuously recognising
the sameindividual across time and camera views. Researchers of pedestrian ReID
and theirGPUs spend enormous energy producing novel algorithms, challenging
datasets,and readily accessible tools to successfully improve results on
standard metrics.Yet practitioners in biometrics, surveillance, and autonomous
driving have not re-alized benefits that reflect these metrics. Different
detections, slight occlusions,changes in perspective, and other banal
perturbations render the best neural net-works virtually useless. This work
makes two contributions. First, we introducethe ReID community to a budding
area of computer vision research in model eval-uation. By adapting established
principles of psychophysical evaluation from psy-chology, we can quantify the
performance degradation and begin research thatwill improve the utility of
pedestrian ReID models; not just their performance ontest sets. Second, we
introduce NuscenesReID, a challenging new ReID datasetdesigned to reflect the
real world autonomous vehicle conditions in which ReIDalgorithms are used. We
show that, despite performing well on existing ReIDdatasets, most models are
not robust to synthetic augmentations or to the morerealistic NuscenesReID
data.
