  Encoded text representations often capture sensitive attributes about
individuals (e.g., race or gender), which raise privacy concerns and can make
downstream models unfair to certain groups. In this work, we propose FEDERATE,
an approach that combines ideas from differential privacy and adversarial
training to learn private text representations which also induces fairer
models. We empirically evaluate the trade-off between the privacy of the
representations and the fairness and accuracy of the downstream model on four
NLP datasets. Our results show that FEDERATE consistently improves upon
previous methods, and thus suggest that privacy and fairness can positively
reinforce each other.
