  We present FairX, an open-source Python-based benchmarking tool designed for
the comprehensive analysis of models under the umbrella of fairness, utility,
and eXplainability (XAI). FairX enables users to train benchmarking
bias-removal models and evaluate their fairness using a wide array of fairness
metrics, data utility metrics, and generate explanations for model predictions,
all within a unified framework. Existing benchmarking tools do not have the way
to evaluate synthetic data generated from fair generative models, also they do
not have the support for training fair generative models either. In FairX, we
add fair generative models in the collection of our fair-model library
(pre-processing, in-processing, post-processing) and evaluation metrics for
evaluating the quality of synthetic fair data. This version of FairX supports
both tabular and image datasets. It also allows users to provide their own
custom datasets. The open-source FairX benchmarking package is publicly
available at https://github.com/fahim-sikder/FairX.
