  Bias in training datasets must be managed for various groups in
classification tasks to ensure parity or equal treatment. With the recent
growth in artificial intelligence models and their expanding role in automated
decision-making, ensuring that these models are not biased is vital. There is
an abundance of evidence suggesting that these models could contain or even
amplify the bias present in the data on which they are trained, inherent to
their objective function and learning algorithms; Many researchers direct their
attention to this issue in different directions, namely, changing data to be
statistically independent, adversarial training for restricting the
capabilities of a particular competitor who aims to maximize parity, etc. These
methods result in information loss and do not provide a suitable balance
between accuracy and fairness or do not ensure limiting the biases in training.
To this end, we propose a powerful strategy for training deep learning models
called the Distraction module, which can be theoretically proven effective in
controlling bias from affecting the classification results. This method can be
utilized with different data types (e.g., Tabular, images, graphs, etc.). We
demonstrate the potency of the proposed method by testing it on UCI Adult and
Heritage Health datasets (tabular), POKEC-Z, POKEC-N and NBA datasets (graph),
and CelebA dataset (vision). Using state-of-the-art methods proposed in the
fairness literature for each dataset, we exhibit our model is superior to these
proposed methods in minimizing bias and maintaining accuracy.
