  Appearance of a face can be greatly altered by growing a beard and mustache.
The facial hairstyles in a pair of images can cause marked changes to the
impostor distribution and the genuine distribution. Also, different
distributions of facial hairstyle across demographics could cause a false
impression of relative accuracy across demographics. We first show that, even
though larger training sets boost the recognition accuracy on all facial
hairstyles, accuracy variations caused by facial hairstyles persist regardless
of the size of the training set. Then, we analyze the impact of having
different fractions of the training data represent facial hairstyles. We
created balanced training sets using a set of identities available in
Webface42M that both have clean-shaven and facial hair images. We find that,
even when a face recognition model is trained with a balanced clean-shaven /
facial hair training set, accuracy variation on the test data does not
diminish. Next, data augmentation is employed to further investigate the effect
of facial hair distribution in training data by manipulating facial hair pixels
with the help of facial landmark points and a facial hair segmentation model.
Our results show facial hair causes an accuracy gap between clean-shaven and
facial hair images, and this impact can be significantly different between
African-Americans and Caucasians.
