  Originating from the diffusion phenomenon in physics that describes particle
movement, the diffusion generative models inherit the characteristics of
stochastic random walk in the data space along the denoising trajectory.
However, the intrinsic mutual interference among image regions contradicts the
need for practical downstream application scenarios where the preservation of
low-level pixel information from given conditioning is desired (e.g.,
customization tasks like personalized generation and inpainting based on a
user-provided single image). In this work, we investigate the diffusion
(physics) in diffusion (machine learning) properties and propose our Cyclic
One-Way Diffusion (COW) method to control the direction of diffusion phenomenon
given a pre-trained frozen diffusion model for versatile customization
application scenarios, where the low-level pixel information from the
conditioning needs to be preserved. Notably, unlike most current methods that
incorporate additional conditions by fine-tuning the base text-to-image
diffusion model or learning auxiliary networks, our method provides a novel
perspective to understand the task needs and is applicable to a wider range of
customization scenarios in a learning-free manner. Extensive experiment results
show that our proposed COW can achieve more flexible customization based on
strict visual conditions in different application settings. Project page:
https://wangruoyu02.github.io/cow.github.io/.
