  Diffusion models have been extensively used in data generation tasks and are
recognized as one of the best generative models. However, their time-consuming
deployment, long inference time, and requirements on large memory limit their
application on mobile devices. In this paper, we propose a method based on the
improved Straight-Through Estimator to improve the deployment efficiency of
diffusion models. Specifically, we add sparse masks to the Convolution and
Linear layers in a pre-trained diffusion model, then use design progressive
sparsity for model training in the fine-tuning stage, and switch the inference
mask on and off, which supports a flexible choice of sparsity during inference
according to the FID and MACs requirements. Experiments on four datasets
conducted on a state-of-the-art Transformer-based diffusion model demonstrate
that our method reduces MACs by $50\%$ while increasing FID by only 1.5 on
average. Under other MACs conditions, the FID is also lower than 1$\sim$137
compared to other methods.
