  Biometric data contains distinctive human traits such as facial features or
gait patterns. The use of biometric data permits an individuation so exact that
the data is utilized effectively in identification and authentication systems.
But for this same reason, privacy protections become indispensably necessary.
Privacy protection is extensively afforded by the technique of anonymization.
Anonymization techniques protect sensitive personal data from biometrics by
obfuscating or removing information that allows linking records to the
generating individuals, to achieve high levels of anonymity. However, our
understanding and possibility to develop effective anonymization relies, in
equal parts, on the effectiveness of the methods employed to evaluate
anonymization performance. In this paper, we assess the state-of-the-art
methods used to evaluate the performance of anonymization techniques for facial
images and for gait patterns. We demonstrate that the state-of-the-art
evaluation methods have serious and frequent shortcomings. In particular, we
find that the underlying assumptions of the state-of-the-art are quite
unwarranted. State-of-the-art methods generally assume a difficult recognition
scenario and thus a weak adversary. However, that assumption causes
state-of-the-art evaluations to grossly overestimate the performance of the
anonymization. Therefore, we propose a strong adversary which is aware of the
anonymization in place. We improve the selection process for the evaluation
dataset, and we reduce the numbers of identities contained in the dataset while
ensuring that these identities remain easily distinguishable from one another.
Our novel evaluation methodology surpasses the state-of-the-art because we
measure worst-case performance and so deliver a highly reliable evaluation of
biometric anonymization techniques.
