  Denoising Diffusion Probabilistic Model (DDPM) is able to make flexible
conditional image generation from prior noise to real data, by introducing an
independent noise-aware classifier to provide conditional gradient guidance at
each time step of denoising process. However, due to the ability of classifier
to easily discriminate an incompletely generated image only with high-level
structure, the gradient, which is a kind of class information guidance, tends
to vanish early, leading to the collapse from conditional generation process
into the unconditional process. To address this problem, we propose two simple
but effective approaches from two perspectives. For sampling procedure, we
introduce the entropy of predicted distribution as the measure of guidance
vanishing level and propose an entropy-aware scaling method to adaptively
recover the conditional semantic guidance. For training stage, we propose the
entropy-aware optimization objectives to alleviate the overconfident prediction
for noisy data.On ImageNet1000 256x256, with our proposed sampling scheme and
trained classifier, the pretrained conditional and unconditional DDPM model can
achieve 10.89% (4.59 to 4.09) and 43.5% (12 to 6.78) FID improvement
respectively. The code is available at https://github.com/ZGCTroy/ED-DPM.
