  In this work, we study the generalization properties of diffusion models in a
few-shot setup, introduce a novel tuning-free paradigm to synthesize the target
out-of-domain (OOD) data, and demonstrate its advantages compared to existing
methods in data-sparse scenarios with large domain gaps. Specifically, given a
pre-trained model and a small set of images that are OOD relative to the
model's training distribution, we explore whether the frozen model is able to
generalize to this new domain. We begin by revealing that Denoising Diffusion
Probabilistic Models (DDPMs) trained on single-domain images are already
equipped with sufficient representation abilities to reconstruct arbitrary
images from the inverted latent encoding following bi-directional deterministic
diffusion and denoising trajectories. We then demonstrate through both
theoretical and empirical perspectives that the OOD images establish Gaussian
priors in latent spaces of the given model, and the inverted latent modes are
separable from their initial training domain. We then introduce our novel
tuning-free paradigm to synthesize new images of the target unseen domain by
discovering qualified OOD latent encodings in the inverted noisy spaces. This
is fundamentally different from the current paradigm that seeks to modify the
denoising trajectory to achieve the same goal by tuning the model parameters.
Extensive cross-model and domain experiments show that our proposed method can
expand the latent space and generate unseen images via frozen DDPMs without
impairing the quality of generation of their original domain. We also showcase
a practical application of our proposed heuristic approach in dramatically
different domains using astrophysical data, revealing the great potential of
such a generalization paradigm in data spare fields such as scientific
explorations.
