  Most convolutional network (CNN)-based inpainting methods adopt standard
convolution to indistinguishably treat valid pixels and holes, making them
limited in handling irregular holes and more likely to generate inpainting
results with color discrepancy and blurriness. Partial convolution has been
suggested to address this issue, but it adopts handcrafted feature
re-normalization, and only considers forward mask-updating. In this paper, we
present a learnable attention map module for learning feature renormalization
and mask-updating in an end-to-end manner, which is effective in adapting to
irregular holes and propagation of convolution layers. Furthermore, learnable
reverse attention maps are introduced to allow the decoder of U-Net to
concentrate on filling in irregular holes instead of reconstructing both holes
and known regions, resulting in our learnable bidirectional attention maps.
Qualitative and quantitative experiments show that our method performs
favorably against state-of-the-arts in generating sharper, more coherent and
visually plausible inpainting results. The source code and pre-trained models
will be available.
