  Mitigating biases in generative AI and, particularly in text-to-image models,
is of high importance given their growing implications in society. The biased
datasets used for training pose challenges in ensuring the responsible
development of these models, and mitigation through hard prompting or embedding
alteration, are the most common present solutions. Our work introduces a novel
approach to achieve diverse and inclusive synthetic images by learning a
direction in the latent space and solely modifying the initial Gaussian noise
provided for the diffusion process. Maintaining a neutral prompt and untouched
embeddings, this approach successfully adapts to diverse debiasing scenarios,
such as geographical biases. Moreover, our work proves it is possible to
linearly combine these learned latent directions to introduce new mitigations,
and if desired, integrate it with text embedding adjustments. Furthermore,
text-to-image models lack transparency for assessing bias in outputs, unless
visually inspected. Thus, we provide a tool to empower developers to select
their desired concepts to mitigate. The project page with code is available
online.
