  Classifier-free guidance (CFG) is a fundamental tool in modern diffusion
models for text-guided generation. Although effective, CFG has notable
drawbacks. For instance, DDIM with CFG lacks invertibility, complicating image
editing; furthermore, high guidance scales, essential for high-quality outputs,
frequently result in issues like mode collapse. Contrary to the widespread
belief that these are inherent limitations of diffusion models, this paper
reveals that the problems actually stem from the off-manifold phenomenon
associated with CFG, rather than the diffusion models themselves. More
specifically, inspired by the recent advancements of diffusion model-based
inverse problem solvers (DIS), we reformulate text-guidance as an inverse
problem with a text-conditioned score matching loss, and develop CFG++, a novel
approach that tackles the off-manifold challenges inherent in traditional CFG.
CFG++ features a surprisingly simple fix to CFG, yet it offers significant
improvements, including better sample quality for text-to-image generation,
invertibility, smaller guidance scales, reduced mode collapse, etc.
Furthermore, CFG++ enables seamless interpolation between unconditional and
conditional sampling at lower guidance scales, consistently outperforming
traditional CFG at all scales. Experimental results confirm that our method
significantly enhances performance in text-to-image generation, DDIM inversion,
editing, and solving inverse problems, suggesting a wide-ranging impact and
potential applications in various fields that utilize text guidance. Project
Page: https://cfgpp-diffusion.github.io/.
