  Face recognition is known to exhibit bias - subjects in a certain demographic
group can be better recognized than other groups. This work aims to learn a
fair face representation, where faces of every group could be more equally
represented. Our proposed group adaptive classifier mitigates bias by using
adaptive convolution kernels and attention mechanisms on faces based on their
demographic attributes. The adaptive module comprises kernel masks and
channel-wise attention maps for each demographic group so as to activate
different facial regions for identification, leading to more discriminative
features pertinent to their demographics. Our introduced automated adaptation
strategy determines whether to apply adaptation to a certain layer by
iteratively computing the dissimilarity among demographic-adaptive parameters.
A new de-biasing loss function is proposed to mitigate the gap of average
intra-class distance between demographic groups. Experiments on face benchmarks
(RFW, LFW, IJB-A, and IJB-C) show that our work is able to mitigate face
recognition bias across demographic groups while maintaining the competitive
accuracy.
