  The rapid evolution of the fashion industry increasingly intersects with
technological advancements, particularly through the integration of generative
AI. This study introduces a novel generative pipeline designed to transform the
fashion design process by employing latent diffusion models. Utilizing
ControlNet and LoRA fine-tuning, our approach generates high-quality images
from multimodal inputs such as text and sketches. We leverage and enhance
state-of-the-art virtual try-on datasets, including Multimodal Dress Code and
VITON-HD, by integrating sketch data. Our evaluation, utilizing metrics like
FID, CLIP Score, and KID, demonstrates that our model significantly outperforms
traditional stable diffusion models. The results not only highlight the
effectiveness of our model in generating fashion-appropriate outputs but also
underscore the potential of diffusion models in revolutionizing fashion design
workflows. This research paves the way for more interactive, personalized, and
technologically enriched methodologies in fashion design and representation,
bridging the gap between creative vision and practical application.
