  Image inpainting approaches have achieved significant progress with the help
of deep neural networks. However, existing approaches mainly focus on
leveraging the priori distribution learned by neural networks to produce a
single inpainting result or further yielding multiple solutions, where the
controllability is not well studied. This paper develops a novel image
inpainting approach that enables users to customize the inpainting result by
their own preference or memory. Specifically, our approach is composed of two
stages that utilize the prior of neural network and user's guidance to jointly
inpaint corrupted images. In the first stage, an autoencoder based on a novel
external spatial attention mechanism is deployed to produce reconstructed
features of the corrupted image and a coarse inpainting result that provides
semantic mask as the medium for user interaction. In the second stage, a
semantic decoder that takes the reconstructed features as prior is adopted to
synthesize a fine inpainting result guided by user's customized semantic mask,
so that the final inpainting result will share the same content with user's
guidance while the textures and colors reconstructed in the first stage are
preserved. Extensive experiments demonstrate the superiority of our approach in
terms of inpainting quality and controllability.
