  Recent deep learning based image inpainting methods which utilize contextual
information and two-stage architecture have exhibited remarkable performance.
However, the two-stage architecture is time-consuming, the contextual
information lack high-level semantics and ignores both the semantic relevance
and distance information of hole's feature patches, these limitations result in
blurry textures and distorted structures of final result. Motivated by these
observations, we propose a new deep generative model-based approach, which
trains a shared network twice with different targets and utilizes a single
network during the testing phase, so that we can effectively save inference
time. Specifically, the targets of two training steps are structure
reconstruction and texture generation respectively. During the second training,
we first propose a Pyramid Filling Block (PF-block) to utilize the high-level
features that the hole regions has been filled to guide the filling process of
low-level features progressively, the missing content can be filled from deep
to shallow in a pyramid fashion. Then, inspired by the classical bilateral
filter [30], we propose the Bilateral Attention layer (BA-layer) to optimize
filled feature map, which synthesizes feature patches at each position by
computing weighted sums of the surrounding feature patches, these weights are
derived by considering both distance and value relationships between feature
patches, thus making the visually plausible inpainting results. Finally,
experiments on multiple publicly available datasets show the superior
performance of our approach.
