  We present OxonFair, a new open source toolkit for enforcing fairness in
binary classification. Compared to existing toolkits: (i) We support NLP and
Computer Vision classification as well as standard tabular problems. (ii) We
support enforcing fairness on validation data, making us robust to a wide range
of overfitting challenges. (iii) Our approach can optimize any measure based on
True Positives, False Positive, False Negatives, and True Negatives. This makes
it easily extendable and much more expressive than existing toolkits. It
supports 9/9 and 10/10 of the decision-based group metrics of two popular
review papers. (iv) We jointly optimize a performance objective. This not only
minimizes degradation while enforcing fairness, but can improve the performance
of otherwise inadequately tuned unfair baselines. OxonFair is compatible with
standard ML toolkits including sklearn, Autogluon, and PyTorch and is available
online at https://github.com/oxfordinternetinstitute/oxonfair
