  Fair representations are a powerful tool for establishing criteria like
statistical parity, proxy non-discrimination, and equality of opportunity in
learned models. Existing techniques for learning these representations are
typically model-agnostic, as they preprocess the original data such that the
output satisfies some fairness criterion, and can be used with arbitrary
learning methods. In contrast, we demonstrate the promise of learning a
model-aware fair representation, focusing on kernel-based models. We leverage
the classical Sufficient Dimension Reduction (SDR) framework to construct
representations as subspaces of the reproducing kernel Hilbert space (RKHS),
whose member functions are guaranteed to satisfy fairness. Our method supports
several fairness criteria, continuous and discrete data, and multiple protected
attributes. We further show how to calibrate the accuracy tradeoff by
characterizing it in terms of the principal angles between subspaces of the
RKHS. Finally, we apply our approach to obtain the first Fair Gaussian Process
(FGP) prior for fair Bayesian learning, and show that it is competitive with,
and in some cases outperforms, state-of-the-art methods on real data.
