  Diffusion models have shown remarkable capabilities in generating high
quality and creative images conditioned on text. An interesting application of
such models is structure preserving text guided image editing. Existing
approaches rely on text conditioned diffusion models such as Stable Diffusion
or Imagen and require compute intensive optimization of text embeddings or
fine-tuning the model weights for text guided image editing. We explore text
guided image editing with a Hybrid Diffusion Model (HDM) architecture similar
to DALLE-2. Our architecture consists of a diffusion prior model that generates
CLIP image embedding conditioned on a text prompt and a custom Latent Diffusion
Model trained to generate images conditioned on CLIP image embedding. We
discover that the diffusion prior model can be used to perform text guided
conceptual edits on the CLIP image embedding space without any finetuning or
optimization. We combine this with structure preserving edits on the image
decoder using existing approaches such as reverse DDIM to perform text guided
image editing. Our approach, PRedItOR does not require additional inputs,
fine-tuning, optimization or objectives and shows on par or better results than
baselines qualitatively and quantitatively. We provide further analysis and
understanding of the diffusion prior model and believe this opens up new
possibilities in diffusion models research.
