  Diffusion-based video generation has achieved significant progress, yet
generating multiple actions that occur sequentially remains a formidable task.
Directly generating a video with sequential actions can be extremely
challenging due to the scarcity of fine-grained action annotations and the
difficulty in establishing temporal semantic correspondences and maintaining
long-term consistency. To tackle this, we propose an intuitive and
straightforward solution: splicing multiple single-action video segments
sequentially. The core challenge lies in generating smooth and natural
transitions between these segments given the inherent complexity and
variability of action transitions. We introduce MAVIN (Multi-Action Video
INfilling model), designed to generate transition videos that seamlessly
connect two given videos, forming a cohesive integrated sequence. MAVIN
incorporates several innovative techniques to address challenges in the
transition video infilling task. Firstly, a consecutive noising strategy
coupled with variable-length sampling is employed to handle large infilling
gaps and varied generation lengths. Secondly, boundary frame guidance (BFG) is
proposed to address the lack of semantic guidance during transition generation.
Lastly, a Gaussian filter mixer (GFM) dynamically manages noise initialization
during inference, mitigating train-test discrepancy while preserving generation
flexibility. Additionally, we introduce a new metric, CLIP-RS (CLIP Relative
Smoothness), to evaluate temporal coherence and smoothness, complementing
traditional quality-based metrics. Experimental results on horse and tiger
scenarios demonstrate MAVIN's superior performance in generating smooth and
coherent video transitions compared to existing methods.
