  Convolutional neural networks (CNNs) have achieved superhuman performance in
multiple vision tasks, especially image classification. However, unlike humans,
CNNs leverage spurious features, such as background information to make
decisions. This tendency creates different problems in terms of robustness or
weak generalization performance. Through our work, we introduce a contrastive
learning-based approach (CLAD) to mitigate the background bias in CNNs. CLAD
encourages semantic focus on object foregrounds and penalizes learning features
from irrelavant backgrounds. Our method also introduces an efficient way of
sampling negative samples. We achieve state-of-the-art results on the
Background Challenge dataset, outperforming the previous benchmark with a
margin of 4.1\%. Our paper shows how CLAD serves as a proof of concept for
debiasing of spurious features, such as background and texture (in
supplementary material).
