  This paper presents an energy-efficient stable diffusion processor for
text-to-image generation. While stable diffusion attained attention for
high-quality image synthesis results, its inherent characteristics hinder its
deployment on mobile platforms. The proposed processor achieves high throughput
and energy efficiency with three key features as solutions: 1) Patch
similarity-based sparsity augmentation (PSSA) to reduce external memory access
(EMA) energy of self-attention score by 60.3 %, leading to 37.8 % total EMA
energy reduction. 2) Text-based important pixel spotting (TIPS) to allow 44.8 %
of the FFN layer workload to be processed with low-precision activation. 3)
Dual-mode bit-slice core (DBSC) architecture to enhance energy efficiency in
FFN layers by 43.0 %. The proposed processor is implemented in 28 nm CMOS
technology and achieves 3.84 TOPS peak throughput with 225.6 mW average power
consumption. In sum, 28.6 mJ/iteration highly energy-efficient text-to-image
generation processor can be achieved at MS-COCO dataset.
