  Deep neural networks (DNNs), despite their impressive ability to generalize
over-capacity networks, often rely heavily on malignant bias as shortcuts
instead of task-related information for discriminative tasks. To address this
problem, recent studies utilize auxiliary information related to the bias,
which is rarely obtainable in practice, or sift through a handful of bias-free
samples for debiasing. However, the success of these methods is not always
guaranteed due to the unfulfilled presumptions. In this paper, we propose a
novel method, Contrastive Debiasing via Generative Bias-transformation (CDvG),
which works without explicit bias labels or bias-free samples. Motivated by our
observation that not only discriminative models but also image translation
models tend to focus on the malignant bias, CDvG employs an image translation
model to transform one bias mode into another while preserving the
task-relevant information. Additionally, the bias-transformed views are set
against each other through contrastive learning to learn bias-invariant
representations. Our method demonstrates superior performance compared to prior
approaches, especially when bias-free samples are scarce or absent.
Furthermore, CDvG can be integrated with the methods that focus on bias-free
samples in a plug-and-play manner for additional enhancements, as demonstrated
by diverse experimental results.
