  Machine learning models have shown exceptional prowess in solving complex
issues across various domains. However, these models can sometimes exhibit
biased decision-making, resulting in unequal treatment of different groups.
Despite substantial research on counterfactual fairness, methods to reduce the
impact of multivariate and continuous sensitive variables on decision-making
outcomes are still underdeveloped. We propose a novel data pre-processing
algorithm, Orthogonal to Bias (OB), which is designed to eliminate the
influence of a group of continuous sensitive variables, thus promoting
counterfactual fairness in machine learning applications. Our approach, based
on the assumption of a jointly normal distribution within a structural causal
model (SCM), demonstrates that counterfactual fairness can be achieved by
ensuring the data is orthogonal to the observed sensitive variables. The OB
algorithm is model-agnostic, making it applicable to a wide range of machine
learning models and tasks. Additionally, it includes a sparse variant to
improve numerical stability through regularization. Empirical evaluations on
both simulated and real-world datasets, encompassing settings with both
discrete and continuous sensitive variables, show that our methodology
effectively promotes fairer outcomes without compromising accuracy.
