  We present TALE, a novel training-free framework harnessing the generative
capabilities of text-to-image diffusion models to address the cross-domain
image composition task that focuses on flawlessly incorporating user-specified
objects into a designated visual contexts regardless of domain disparity.
Previous methods often involve either training auxiliary networks or finetuning
diffusion models on customized datasets, which are expensive and may undermine
the robust textual and visual priors of pre-trained diffusion models. Some
recent works attempt to break the barrier by proposing training-free
workarounds that rely on manipulating attention maps to tame the denoising
process implicitly. However, composing via attention maps does not necessarily
yield desired compositional outcomes. These approaches could only retain some
semantic information and usually fall short in preserving identity
characteristics of input objects or exhibit limited background-object style
adaptation in generated images. In contrast, TALE is a novel method that
operates directly on latent space to provide explicit and effective guidance
for the composition process to resolve these problems. Specifically, we equip
TALE with two mechanisms dubbed Adaptive Latent Manipulation and Energy-guided
Latent Optimization. The former formulates noisy latents conducive to
initiating and steering the composition process by directly leveraging
background and foreground latents at corresponding timesteps, and the latter
exploits designated energy functions to further optimize intermediate latents
conforming to specific conditions that complement the former to generate
desired final results. Our experiments demonstrate that TALE surpasses prior
baselines and attains state-of-the-art performance in image-guided composition
across various photorealistic and artistic domains.
