  Speaker recognition is increasingly used in several everyday applications
including smart speakers, customer care centers and other speech-driven
analytics. It is crucial to accurately evaluate and mitigate biases present in
machine learning (ML) based speech technologies, such as speaker recognition,
to ensure their inclusive adoption. ML fairness studies with respect to various
demographic factors in modern speaker recognition systems are lagging compared
to other human-centered applications such as face recognition. Existing studies
on fairness in speaker recognition systems are largely limited to evaluating
biases at specific operating points of the systems, which can lead to false
expectations of fairness. Moreover, there are only a handful of bias mitigation
strategies developed for speaker recognition systems. In this paper, we
systematically evaluate the biases present in speaker recognition systems with
respect to gender across a range of system operating points. We also propose
adversarial and multi-task learning techniques to improve the fairness of these
systems. We show through quantitative and qualitative evaluations that the
proposed methods improve the fairness of ASV systems over baseline methods
trained using data balancing techniques. We also present a fairness-utility
trade-off analysis to jointly examine fairness and the overall system
performance. We show that although systems trained using adversarial techniques
improve fairness, they are prone to reduced utility. On the other hand,
multi-task methods can improve the fairness while retaining the utility. These
findings can inform the choice of bias mitigation strategies in the field of
speaker recognition.
