  We consider the problem of filling in missing spatio-temporal regions of a
video. We provide a novel flow-based solution by introducing a generative model
of images in relation to the scene (without missing regions) and mappings from
the scene to images. We use the model to jointly infer the scene template, a 2D
representation of the scene, and the mappings. This ensures consistency of the
frame-to-frame flows generated to the underlying scene, reducing geometric
distortions in flow based inpainting. The template is mapped to the missing
regions in the video by a new L2-L1 interpolation scheme, creating crisp
inpaintings and reducing common blur and distortion artifacts. We show on two
benchmark datasets that our approach out-performs state-of-the-art
quantitatively and in user studies.
