  This paper presents Video-P2P, a novel framework for real-world video editing
with cross-attention control. While attention control has proven effective for
image editing with pre-trained image generation models, there are currently no
large-scale video generation models publicly available. Video-P2P addresses
this limitation by adapting an image generation diffusion model to complete
various video editing tasks. Specifically, we propose to first tune a
Text-to-Set (T2S) model to complete an approximate inversion and then optimize
a shared unconditional embedding to achieve accurate video inversion with a
small memory cost. For attention control, we introduce a novel
decoupled-guidance strategy, which uses different guidance strategies for the
source and target prompts. The optimized unconditional embedding for the source
prompt improves reconstruction ability, while an initialized unconditional
embedding for the target prompt enhances editability. Incorporating the
attention maps of these two branches enables detailed editing. These technical
designs enable various text-driven editing applications, including word swap,
prompt refinement, and attention re-weighting. Video-P2P works well on
real-world videos for generating new characters while optimally preserving
their original poses and scenes. It significantly outperforms previous
approaches.
