  Video moment retrieval and highlight detection have received attention in the
current era of video content proliferation, aiming to localize moments and
estimate clip relevances based on user-specific queries. Given that the video
content is continuous in time, there is often a lack of clear boundaries
between temporal events in a video. This boundary ambiguity makes it
challenging for the model to learn text-video clip correspondences, resulting
in the subpar performance of existing methods in predicting target segments. To
alleviate this problem, we propose to solve the two tasks jointly from the
perspective of denoising generation. Moreover, the target boundary can be
localized clearly by iterative refinement from coarse to fine. Specifically, a
novel framework, DiffusionVMR, is proposed to redefine the two tasks as a
unified conditional denoising generation process by combining the diffusion
model. During training, Gaussian noise is added to corrupt the ground truth,
with noisy candidates produced as input. The model is trained to reverse this
noise addition process. In the inference phase, DiffusionVMR initiates directly
from Gaussian noise and progressively refines the proposals from the noise to
the meaningful output. Notably, the proposed DiffusionVMR inherits the
advantages of diffusion models that allow for iteratively refined results
during inference, enhancing the boundary transition from coarse to fine.
Furthermore, the training and inference of DiffusionVMR are decoupled. An
arbitrary setting can be used in DiffusionVMR during inference without
consistency with the training phase. Extensive experiments conducted on five
widely-used benchmarks (i.e., QVHighlight, Charades-STA, TACoS,
YouTubeHighlights and TVSum) across two tasks (moment retrieval and/or
highlight detection) demonstrate the effectiveness and flexibility of the
proposed DiffusionVMR.
