  We tackle societal bias in image-text datasets by removing spurious
correlations between protected groups and image attributes. Traditional methods
only target labeled attributes, ignoring biases from unlabeled ones. Using
text-guided inpainting models, our approach ensures protected group
independence from all attributes and mitigates inpainting biases through data
filtering. Evaluations on multi-label image classification and image captioning
tasks show our method effectively reduces bias without compromising performance
across various models.
