  Applying pre-trained generative denoising diffusion models (DDMs) for
downstream tasks such as image semantic editing usually requires either
fine-tuning DDMs or learning auxiliary editing networks in the existing
literature. In this work, we present our BoundaryDiffusion method for
efficient, effective and light-weight semantic control with frozen pre-trained
DDMs, without learning any extra networks. As one of the first learning-free
diffusion editing works, we start by seeking a comprehensive understanding of
the intermediate high-dimensional latent spaces by theoretically and
empirically analyzing their probabilistic and geometric behaviors in the Markov
chain. We then propose to further explore the critical step for editing in the
denoising trajectory that characterizes the convergence of a pre-trained DDM
and introduce an automatic search method. Last but not least, in contrast to
the conventional understanding that DDMs have relatively poor semantic
behaviors, we prove that the critical latent space we found already exhibits
semantic subspace boundaries at the generic level in unconditional DDMs, which
allows us to do controllable manipulation by guiding the denoising trajectory
towards the targeted boundary via a single-step operation. We conduct extensive
experiments on multiple DPMs architectures (DDPM, iDDPM) and datasets (CelebA,
CelebA-HQ, LSUN-church, LSUN-bedroom, AFHQ-dog) with different resolutions (64,
256), achieving superior or state-of-the-art performance in various task
scenarios (image semantic editing, text-based editing, unconditional semantic
control) to demonstrate the effectiveness.
