  Current AI regulations require discarding sensitive features (e.g., gender,
race, religion) in the algorithm's decision-making process to prevent unfair
outcomes. However, even without sensitive features in the training set,
algorithms can persist in discrimination. Indeed, when sensitive features are
omitted (fairness under unawareness), they could be inferred through non-linear
relations with the so called proxy features. In this work, we propose a way to
reveal the potential hidden bias of a machine learning model that can persist
even when sensitive features are discarded. This study shows that it is
possible to unveil whether the black-box predictor is still biased by
exploiting counterfactual reasoning. In detail, when the predictor provides a
negative classification outcome, our approach first builds counterfactual
examples for a discriminated user category to obtain a positive outcome. Then,
the same counterfactual samples feed an external classifier (that targets a
sensitive feature) that reveals whether the modifications to the user
characteristics needed for a positive outcome moved the individual to the
non-discriminated group. When this occurs, it could be a warning sign for
discriminatory behavior in the decision process. Furthermore, we leverage the
deviation of counterfactuals from the original sample to determine which
features are proxies of specific sensitive information. Our experiments show
that, even if the model is trained without sensitive features, it often suffers
discriminatory biases.
