  Language models excel at generating coherent text, and model compression
techniques such as knowledge distillation have enabled their use in
resource-constrained settings. However, these models can be biased in multiple
ways, including the unfounded association of male and female genders with
gender-neutral professions. Therefore, knowledge distillation without any
fairness constraints may preserve or exaggerate the teacher model's biases onto
the distilled model. To this end, we present a novel approach to mitigate
gender disparity in text generation by learning a fair model during knowledge
distillation. We propose two modifications to the base knowledge distillation
based on counterfactual role reversal$\unicode{x2014}$modifying teacher
probabilities and augmenting the training set. We evaluate gender polarity
across professions in open-ended text generated from the resulting distilled
and finetuned GPT$\unicode{x2012}$2 models and demonstrate a substantial
reduction in gender disparity with only a minor compromise in utility. Finally,
we observe that language models that reduce gender polarity in language
generation do not improve embedding fairness or downstream classification
fairness.
