  This work revisits the ChaLearn First Impressions database, annotated for
personality perception using pairwise comparisons via crowdsourcing. We analyse
for the first time the original pairwise annotations, and reveal existing
person perception biases associated to perceived attributes like gender,
ethnicity, age and face attractiveness. We show how person perception bias can
influence data labelling of a subjective task, which has received little
attention from the computer vision and machine learning communities by now. We
further show that the mechanism used to convert pairwise annotations to
continuous values may magnify the biases if no special treatment is considered.
The findings of this study are relevant for the computer vision community that
is still creating new datasets on subjective tasks, and using them for
practical applications, ignoring these perceptual biases.
