  Domain Generalization techniques aim to enhance model robustness by
simulating novel data distributions during training, typically through various
augmentation or stylization strategies. However, these methods frequently
suffer from limited control over the diversity of generated images and lack
assurance that these images span distinct distributions. To address these
challenges, we propose FDS, Feedback-guided Domain Synthesis, a novel strategy
that employs diffusion models to synthesize novel, pseudo-domains by training a
single model on all source domains and performing domain mixing based on
learned features. By incorporating images that pose classification challenges
to models trained on original samples, alongside the original dataset, we
ensure the generation of a training set that spans a broad distribution
spectrum. Our comprehensive evaluations demonstrate that this methodology sets
new benchmarks in domain generalization performance across a range of
challenging datasets, effectively managing diverse types of domain shifts. The
implementation is available at: \url{https://github.com/Mehrdad-Noori/FDS.git}.
