  With the ever-growing popularity of Artificial Intelligence, there is an
increasing demand for more performant and efficient underlying hardware.
Convolutional Neural Networks (CNN) are a workload of particular importance,
which achieve high accuracy in computer vision applications. Inside CNNs, a
significant number of the post-activation values are zero, resulting in many
redundant computations. Recent works have explored this post-activation
sparsity on instruction-based CNN accelerators but not on streaming CNN
accelerators, despite the fact that streaming architectures are considered the
leading design methodology in terms of performance. In this paper, we highlight
the challenges associated with exploiting post-activation sparsity for
performance gains in streaming CNN accelerators, and demonstrate our approach
to address them. Using a set of modern CNN benchmarks, our streaming sparse
accelerators achieve 1.41x to 1.93x efficiency (GOP/s/DSP) compared to
state-of-the-art instruction-based sparse accelerators.
