  Dataset bias is a significant problem in training fair classifiers. When
attributes unrelated to classification exhibit strong biases towards certain
classes, classifiers trained on such dataset may overfit to these bias
attributes, substantially reducing the accuracy for minority groups. Mitigation
techniques can be categorized according to the availability of bias information
(\ie, prior knowledge). Although scenarios with unknown biases are better
suited for real-world settings, previous work in this field often suffers from
a lack of interpretability regarding biases and lower performance. In this
study, we propose a framework to identify potential biases as keywords without
prior knowledge based on the partial occurrence in the captions. We further
propose two debiasing methods: (a) handing over to an existing debiasing
approach which requires prior knowledge by assigning pseudo-labels, and (b)
employing data augmentation via text-to-image generative models, using acquired
bias keywords as prompts. Despite its simplicity, experimental results show
that our framework not only outperforms existing methods without prior
knowledge, but also is even comparable with a method that assumes prior
knowledge.
