  In the world of deep learning, Transformer models have become very
significant, leading to improvements in many areas from understanding language
to recognizing images, covering a wide range of applications. Despite their
success, the deployment of these models in real-time applications, particularly
on edge devices, poses significant challenges due to their quadratic
computational intensity and memory demands. To overcome these challenges we
introduce a novel Hybrid Dynamic Pruning (HDP), an efficient
algorithm-architecture co-design approach that accelerates transformers using
head sparsity, block sparsity and approximation opportunities to reduce
computations in attention and reduce memory access. With the observation of the
huge redundancy in attention scores and attention heads, we propose a novel
integer-based row-balanced block pruning to prune unimportant blocks in the
attention matrix at run time, also propose integer-based head pruning to detect
and prune unimportant heads at an early stage at run time. Also we propose an
approximation method that reduces attention computations. To efficiently
support these methods with lower latency and power efficiency, we propose a HDP
co-processor architecture.
