  Vanilla image completion approaches exhibit sensitivity to large missing
regions, attributed to the limited availability of reference information for
plausible generation. To mitigate this, existing methods incorporate the extra
cue as a guidance for image completion. Despite improvements, these approaches
are often restricted to employing a single modality (e.g., segmentation or
sketch maps), which lacks scalability in leveraging multi-modality for more
plausible completion. In this paper, we propose a novel, simple yet effective
method for Multi-modal Guided Image Completion, dubbed MaGIC, which not only
supports a wide range of single modality as the guidance (e.g., text, canny
edge, sketch, segmentation, depth, and pose), but also adapts to arbitrarily
customized combination of these modalities (i.e., arbitrary multi-modality) for
image completion. For building MaGIC, we first introduce a modality-specific
conditional U-Net (MCU-Net) that injects single-modal signal into a U-Net
denoiser for single-modal guided image completion. Then, we devise a consistent
modality blending (CMB) method to leverage modality signals encoded in multiple
learned MCU-Nets through gradient guidance in latent space. Our CMB is
training-free, thereby avoids the cumbersome joint re-training of different
modalities, which is the secret of MaGIC to achieve exceptional flexibility in
accommodating new modalities for completion. Experiments show the superiority
of MaGIC over state-of-the-art methods and its generalization to various
completion tasks. Our project with code and models is available at
yeates.github.io/MaGIC-Page/.
