  Despite a surge of recent advances in promoting machine Learning (ML)
fairness, the existing mainstream approaches mostly require retraining or
finetuning the entire weights of the neural network to meet the fairness
criteria. However, this is often infeasible in practice for those large-scale
trained models due to large computational and storage costs, low data
efficiency, and model privacy issues. In this paper, we propose a new generic
fairness learning paradigm, called FairReprogram, which incorporates the model
reprogramming technique. Specifically, FairReprogram considers the case where
models can not be changed and appends to the input a set of perturbations,
called the fairness trigger, which is tuned towards the fairness criteria under
a min-max formulation. We further introduce an information-theoretic framework
that explains why and under what conditions fairness goals can be achieved
using the fairness trigger. We show both theoretically and empirically that the
fairness trigger can effectively obscure demographic biases in the output
prediction of fixed ML models by providing false demographic information that
hinders the model from utilizing the correct demographic information to make
the prediction. Extensive experiments on both NLP and CV datasets demonstrate
that our method can achieve better fairness improvements than retraining-based
methods with far less data dependency under two widely-used fairness criteria.
Codes are available at
https://github.com/UCSB-NLP-Chang/Fairness-Reprogramming.git.
