  Despite the high-quality results of text-to-image generation, stereotypical
biases have been spotted in their generated contents, compromising the fairness
of generative models. In this work, we propose to learn adaptive inclusive
tokens to shift the attribute distribution of the final generative outputs.
Unlike existing de-biasing approaches, our method requires neither explicit
attribute specification nor prior knowledge of the bias distribution.
Specifically, the core of our method is a lightweight adaptive mapping network,
which can customize the inclusive tokens for the concepts to be de-biased,
making the tokens generalizable to unseen concepts regardless of their original
bias distributions. This is achieved by tuning the adaptive mapping network
with a handful of balanced and inclusive samples using an anchor loss.
Experimental results demonstrate that our method outperforms previous bias
mitigation methods without attribute specification while preserving the
alignment between generative results and text descriptions. Moreover, our
method achieves comparable performance to models that require specific
attributes or editing directions for generation. Extensive experiments showcase
the effectiveness of our adaptive inclusive tokens in mitigating stereotypical
bias in text-to-image generation. The code will be available at
https://github.com/itsmag11/AITTI.
