  Diffusion models have emerged as powerful generative tools, rivaling GANs in
sample quality and mirroring the likelihood scores of autoregressive models. A
subset of these models, exemplified by DDIMs, exhibit an inherent asymmetry:
they are trained over $T$ steps but only sample from a subset of $T$ during
generation. This selective sampling approach, though optimized for speed,
inadvertently misses out on vital information from the unsampled steps, leading
to potential compromises in sample quality. To address this issue, we present
the S$^{2}$-DMs, which is a new training method by using an innovative
$L_{skip}$, meticulously designed to reintegrate the information omitted during
the selective sampling phase. The benefits of this approach are manifold: it
notably enhances sample quality, is exceptionally simple to implement, requires
minimal code modifications, and is flexible enough to be compatible with
various sampling algorithms. On the CIFAR10 dataset, models trained using our
algorithm showed an improvement of 3.27% to 14.06% over models trained with
traditional methods across various sampling algorithms (DDIMs, PNDMs, DEIS) and
different numbers of sampling steps (10, 20, ..., 1000). On the CELEBA dataset,
the improvement ranged from 8.97% to 27.08%. Access to the code and additional
resources is provided in the github.
