  Denoising diffusion models have emerged as a dominant approach for image
generation, however they still suffer from slow convergence in training and
color shift issues in sampling. In this paper, we identify that these obstacles
can be largely attributed to bias and suboptimality inherent in the default
training paradigm of diffusion models. Specifically, we offer theoretical
insights that the prevailing constant loss weight strategy in
$\epsilon$-prediction of diffusion models leads to biased estimation during the
training phase, hindering accurate estimations of original images. To address
the issue, we propose a simple but effective weighting strategy derived from
the unlocked biased part. Furthermore, we conduct a comprehensive and
systematic exploration, unraveling the inherent bias problem in terms of its
existence, impact and underlying reasons. These analyses contribute to
advancing the understanding of diffusion models. Empirical results demonstrate
that our method remarkably elevates sample quality and displays improved
efficiency in both training and sampling processes, by only adjusting loss
weighting strategy. The code is released publicly at
\url{https://github.com/yuhuUSTC/Debias}
