  We present a novel deep learning approach to synthesize complete face images
in the presence of large ocular region occlusions. This is motivated by recent
surge of VR/AR displays that hinder face-to-face communications. Different from
the state-of-the-art face inpainting methods that have no control over the
synthesized content and can only handle frontal face pose, our approach can
faithfully recover the missing content under various head poses while
preserving the identity. At the core of our method is a novel generative
network with dedicated constraints to regularize the synthesis process. To
preserve the identity, our network takes an arbitrary occlusion-free image of
the target identity to infer the missing content, and its high-level CNN
features as an identity prior to regularize the searching space of generator.
Since the input reference image may have a different pose, a pose map and a
novel pose discriminator are further adopted to supervise the learning of
implicit pose transformations. Our method is capable of generating coherent
facial inpainting with consistent identity over videos with large variations of
head motions. Experiments on both synthesized and real data demonstrate that
our method greatly outperforms the state-of-the-art methods in terms of both
synthesis quality and robustness.
