  Most existing image inpainting algorithms are based on a single view,
struggling with large holes or the holes containing complicated scenes. Some
reference-guided algorithms fill the hole by referring to another viewpoint
image and use 2D image alignment. Due to the camera imaging process, simple 2D
transformation is difficult to achieve a satisfactory result. In this paper, we
propose 3DFill, a simple and efficient method for reference-guided image
inpainting. Given a target image with arbitrary hole regions and a reference
image from another viewpoint, the 3DFill first aligns the two images by a
two-stage method: 3D projection + 2D transformation, which has better results
than 2D image alignment. The 3D projection is an overall alignment between
images and the 2D transformation is a local alignment focused on the hole
region. The entire process of image alignment is self-supervised. We then fill
the hole in the target image with the contents of the aligned image. Finally,
we use a conditional generation network to refine the filled image to obtain
the inpainting result. 3DFill achieves state-of-the-art performance on image
inpainting across a variety of wide view shifts and has a faster inference
speed than other inpainting models.
