  Diffusion models, a specific type of generative model, have achieved
unprecedented performance in recent years and consistently produce high-quality
synthetic samples. A critical prerequisite for their notable success lies in
the presence of a substantial number of training samples, which can be
impractical in real-world applications due to high collection costs or
associated risks. Consequently, various finetuning and regularization
approaches have been proposed to transfer knowledge from existing pre-trained
models to specific target domains with limited data. This paper introduces the
Transfer Guided Diffusion Process (TGDP), a novel approach distinct from
conventional finetuning and regularization methods. We prove that the optimal
diffusion model for the target domain integrates pre-trained diffusion models
on the source domain with additional guidance from a domain classifier. We
further extend TGDP to a conditional version for modeling the joint
distribution of data and its corresponding labels, together with two additional
regularization terms to enhance the model performance. We validate the
effectiveness of TGDP on Gaussian mixture simulations and on real
electrocardiogram (ECG) datasets.
