  Face recognition algorithms, when used in the real world, can be very useful,
but they can also be dangerous when biased toward certain demographics. So, it
is essential to understand how these algorithms are trained and what factors
affect their accuracy and fairness to build better ones. In this study, we shed
some light on the effect of racial distribution in the training data on the
performance of face recognition models. We conduct 16 different experiments
with varying racial distributions of faces in the training data. We analyze
these trained models using accuracy metrics, clustering metrics, UMAP
projections, face quality, and decision thresholds. We show that a uniform
distribution of races in the training datasets alone does not guarantee
bias-free face recognition algorithms and how factors like face image quality
play a crucial role. We also study the correlation between the clustering
metrics and bias to understand whether clustering is a good indicator of bias.
Finally, we introduce a metric called racial gradation to study the inter and
intra race correlation in facial features and how they affect the learning
ability of the face recognition models. With this study, we try to bring more
understanding to an essential element of face recognition training, the data. A
better understanding of the impact of training data on the bias of face
recognition algorithms will aid in creating better datasets and, in turn,
better face recognition systems.
