  Deep neural networks (DNNs) are often prone to learn the spurious
correlations between target classes and bias attributes, like gender and race,
inherent in a major portion of training data (bias-aligned samples), thus
showing unfair behavior and arising controversy in the modern pluralistic and
egalitarian society. In this paper, we propose a novel marginal debiased
network (MDN) to learn debiased representations. More specifically, a marginal
softmax loss (MSL) is designed by introducing the idea of margin penalty into
the fairness problem, which assigns a larger margin for bias-conflicting
samples (data without spurious correlations) than for bias-aligned ones, so as
to deemphasize the spurious correlations and improve generalization on unbiased
test criteria. To determine the margins, our MDN is optimized through a meta
learning framework. We propose a meta equalized loss (MEL) to perceive the
model fairness, and adaptively update the margin parameters by metaoptimization
which requires the trained model guided by the optimal margins should minimize
MEL computed on an unbiased meta-validation set. Extensive experiments on
BiasedMNIST, Corrupted CIFAR-10, CelebA and UTK-Face datasets demonstrate that
our MDN can achieve a remarkable performance on under-represented samples and
obtain superior debiased results against the previous approaches.
