  Image inpainting for completing complicated semantic environments and diverse
hole patterns of corrupted images is challenging even for state-of-the-art
learning-based inpainting methods trained on large-scale data. A reference
image capturing the same scene of a corrupted image offers informative guidance
for completing the corrupted image as it shares similar texture and structure
priors to that of the holes of the corrupted image. In this work, we propose a
transformer-based encoder-decoder network, named TransRef, for reference-guided
image inpainting. Specifically, the guidance is conducted progressively through
a reference embedding procedure, in which the referencing features are
subsequently aligned and fused with the features of the corrupted image. For
precise utilization of the reference features for guidance, a reference-patch
alignment (Ref-PA) module is proposed to align the patch features of the
reference and corrupted images and harmonize their style differences, while a
reference-patch transformer (Ref-PT) module is proposed to refine the embedded
reference feature. Moreover, to facilitate the research of reference-guided
image restoration tasks, we construct a publicly accessible benchmark dataset
containing 50K pairs of input and reference images. Both quantitative and
qualitative evaluations demonstrate the efficacy of the reference information
and the proposed method over the state-of-the-art methods in completing complex
holes. Code and dataset can be accessed at https://github.com/Cameltr/TransRef.
