  For few-shot learning, it is still a critical challenge to realize
photo-realistic face visually dubbing on high-resolution videos. Previous works
fail to generate high-fidelity dubbing results. To address the above problem,
this paper proposes a Deformation Inpainting Network (DINet) for
high-resolution face visually dubbing. Different from previous works relying on
multiple up-sample layers to directly generate pixels from latent embeddings,
DINet performs spatial deformation on feature maps of reference images to
better preserve high-frequency textural details. Specifically, DINet consists
of one deformation part and one inpainting part. In the first part, five
reference facial images adaptively perform spatial deformation to create
deformed feature maps encoding mouth shapes at each frame, in order to align
with the input driving audio and also the head poses of the input source
images. In the second part, to produce face visually dubbing, a feature decoder
is responsible for adaptively incorporating mouth movements from the deformed
feature maps and other attributes (i.e., head pose and upper facial expression)
from the source feature maps together. Finally, DINet achieves face visually
dubbing with rich textural details. We conduct qualitative and quantitative
comparisons to validate our DINet on high-resolution videos. The experimental
results show that our method outperforms state-of-the-art works.
