  Privacy concerns have led to a surge in the creation of synthetic datasets,
with diffusion models emerging as a promising avenue. Although prior studies
have performed empirical evaluations on these models, there has been a gap in
providing a mathematical characterization of their privacy-preserving
capabilities. To address this, we present the pioneering theoretical
exploration of the privacy preservation inherent in discrete diffusion models
(DDMs) for discrete dataset generation. Focusing on per-instance differential
privacy (pDP), our framework elucidates the potential privacy leakage for each
data point in a given training dataset, offering insights into how the privacy
loss of each point correlates with the dataset's distribution. Our bounds also
show that training with $s$-sized data points leads to a surge in privacy
leakage from $(\epsilon, O(\frac{1}{s^2\epsilon}))$-pDP to $(\epsilon,
O(\frac{1}{s\epsilon}))$-pDP of the DDM during the transition from the pure
noise to the synthetic clean data phase, and a faster decay in diffusion
coefficients amplifies the privacy guarantee. Finally, we empirically verify
our theoretical findings on both synthetic and real-world datasets.
