  Structure-guided image completion aims to inpaint a local region of an image
according to an input guidance map from users. While such a task enables many
practical applications for interactive editing, existing methods often struggle
to hallucinate realistic object instances in complex natural scenes. Such a
limitation is partially due to the lack of semantic-level constraints inside
the hole region as well as the lack of a mechanism to enforce realistic object
generation. In this work, we propose a learning paradigm that consists of
semantic discriminators and object-level discriminators for improving the
generation of complex semantics and objects. Specifically, the semantic
discriminators leverage pretrained visual features to improve the realism of
the generated visual concepts. Moreover, the object-level discriminators take
aligned instances as inputs to enforce the realism of individual objects. Our
proposed scheme significantly improves the generation quality and achieves
state-of-the-art results on various tasks, including segmentation-guided
completion, edge-guided manipulation and panoptically-guided manipulation on
Places2 datasets. Furthermore, our trained model is flexible and can support
multiple editing use cases, such as object insertion, replacement, removal and
standard inpainting. In particular, our trained model combined with a novel
automatic image completion pipeline achieves state-of-the-art results on the
standard inpainting task.
