  Removing bias while keeping all task-relevant information is challenging for
fair representation learning methods since they would yield random or
degenerate representations w.r.t. labels when the sensitive attributes
correlate with labels. Existing works proposed to inject the label information
into the learning procedure to overcome such issues. However, the assumption
that the observed labels are clean is not always met. In fact, label bias is
acknowledged as the primary source inducing discrimination. In other words, the
fair pre-processing methods ignore the discrimination encoded in the labels
either during the learning procedure or the evaluation stage. This
contradiction puts a question mark on the fairness of the learned
representations. To circumvent this issue, we explore the following question:
\emph{Can we learn fair representations predictable to latent ideal fair labels
given only access to unreliable labels?} In this work, we propose a
\textbf{D}e-\textbf{B}iased \textbf{R}epresentation Learning for
\textbf{F}airness (DBRF) framework which disentangles the sensitive information
from non-sensitive attributes whilst keeping the learned representations
predictable to ideal fair labels rather than observed biased ones. We formulate
the de-biased learning framework through information-theoretic concepts such as
mutual information and information bottleneck. The core concept is that DBRF
advocates not to use unreliable labels for supervision when sensitive
information benefits the prediction of unreliable labels. Experiment results
over both synthetic and real-world data demonstrate that DBRF effectively
learns de-biased representations towards ideal labels.
