  Video virtual try-on aims to generate realistic sequences that maintain
garment identity and adapt to a person's pose and body shape in source videos.
Traditional image-based methods, relying on warping and blending, struggle with
complex human movements and occlusions, limiting their effectiveness in video
try-on applications. Moreover, video-based models require extensive,
high-quality data and substantial computational resources. To tackle these
issues, we reconceptualize video try-on as a process of generating videos
conditioned on garment descriptions and human motion. Our solution, WildVidFit,
employs image-based controlled diffusion models for a streamlined, one-stage
approach. This model, conditioned on specific garments and individuals, is
trained on still images rather than videos. It leverages diffusion guidance
from pre-trained models including a video masked autoencoder for segment
smoothness improvement and a self-supervised model for feature alignment of
adjacent frame in the latent space. This integration markedly boosts the
model's ability to maintain temporal coherence, enabling more effective video
try-on within an image-based framework. Our experiments on the VITON-HD and
DressCode datasets, along with tests on the VVT and TikTok datasets,
demonstrate WildVidFit's capability to generate fluid and coherent videos. The
project page website is at wildvidfit-project.github.io.
