  Many video editing tasks such as rotoscoping or object removal require the
propagation of context across frames. While transformers and other
attention-based approaches that aggregate features globally have demonstrated
great success at propagating object masks from keyframes to the whole video,
they struggle to propagate high-frequency details such as textures faithfully.
We hypothesize that this is due to an inherent bias of global attention towards
low-frequency features. To overcome this limitation, we present a two-stream
approach, where high-frequency features interact locally and low-frequency
features interact globally. The global interaction stream remains robust in
difficult situations such as large camera motions, where explicit alignment
fails. The local interaction stream propagates high-frequency details through
deformable feature aggregation and, informed by the global interaction stream,
learns to detect and correct errors of the deformation field. We evaluate our
two-stream approach for inpainting tasks, where experiments show that it
improves both the propagation of features within a single frame as required for
image inpainting, as well as their propagation from keyframes to target frames.
Applied to video inpainting, our approach leads to 44% and 26% improvements in
FID and LPIPS scores. Code at https://github.com/runwayml/guided-inpainting
