  Diffusion models have shown great promise in text-guided image style
transfer, but there is a trade-off between style transformation and content
preservation due to their stochastic nature. Existing methods require
computationally expensive fine-tuning of diffusion models or additional neural
network. To address this, here we propose a zero-shot contrastive loss for
diffusion models that doesn't require additional fine-tuning or auxiliary
networks. By leveraging patch-wise contrastive loss between generated samples
and original image embeddings in the pre-trained diffusion model, our method
can generate images with the same semantic content as the source image in a
zero-shot manner. Our approach outperforms existing methods while preserving
content and requiring no additional training, not only for image style transfer
but also for image-to-image translation and manipulation. Our experimental
results validate the effectiveness of our proposed method.
