  Score Distillation Sampling (SDS) is a recent but already widely popular
method that relies on an image diffusion model to control optimization problems
using text prompts. In this paper, we conduct an in-depth analysis of the SDS
loss function, identify an inherent problem with its formulation, and propose a
surprisingly easy but effective fix. Specifically, we decompose the loss into
different factors and isolate the component responsible for noisy gradients. In
the original formulation, high text guidance is used to account for the noise,
leading to unwanted side effects such as oversaturation or repeated detail.
Instead, we train a shallow network mimicking the timestep-dependent frequency
bias of the image diffusion model in order to effectively factor it out. We
demonstrate the versatility and the effectiveness of our novel loss formulation
through qualitative and quantitative experiments, including optimization-based
image synthesis and editing, zero-shot image translation network training, and
text-to-3D synthesis.
