  Video try-on stands as a promising area for its tremendous real-world
potential. Prior works are limited to transferring product clothing images onto
person videos with simple poses and backgrounds, while underperforming on
casually captured videos. Recently, Sora revealed the scalability of Diffusion
Transformer (DiT) in generating lifelike videos featuring real-world scenarios.
Inspired by this, we explore and propose the first DiT-based video try-on
framework for practical in-the-wild applications, named VITON-DiT.
Specifically, VITON-DiT consists of a garment extractor, a Spatial-Temporal
denoising DiT, and an identity preservation ControlNet. To faithfully recover
the clothing details, the extracted garment features are fused with the
self-attention outputs of the denoising DiT and the ControlNet. We also
introduce novel random selection strategies during training and an Interpolated
Auto-Regressive (IAR) technique at inference to facilitate long video
generation. Unlike existing attempts that require the laborious and restrictive
construction of a paired training dataset, severely limiting their scalability,
VITON-DiT alleviates this by relying solely on unpaired human dance videos and
a carefully designed multi-stage training strategy. Furthermore, we curate a
challenging benchmark dataset to evaluate the performance of casual video
try-on. Extensive experiments demonstrate the superiority of VITON-DiT in
generating spatio-temporal consistent try-on results for in-the-wild videos
with complicated human poses.
