  Image generation and editing have seen a great deal of advancements with the
rise of large-scale diffusion models that allow user control of different
modalities such as text, mask, depth maps, etc. However, controlled editing of
videos still lags behind. Prior work in this area has focused on using 2D
diffusion models to globally change the style of an existing video. On the
other hand, in many practical applications, editing localized parts of the
video is critical. In this work, we propose a method to edit videos using a
pre-trained inpainting image diffusion model. We systematically redesign the
forward path of the model by replacing the self-attention modules with an
extended version of attention modules that creates frame-level dependencies. In
this way, we ensure that the edited information will be consistent across all
the video frames no matter what the shape and position of the masked area is.
We qualitatively compare our results with state-of-the-art in terms of accuracy
on several video editing tasks like object retargeting, object replacement, and
object removal tasks. Simulations demonstrate the superior performance of the
proposed strategy.
