  This paper introduces the sparse periodic systolic (SPS) dataflow, which
advances the state-of-the-art hardware accelerator for supporting lightweight
neural networks. Specifically, the SPS dataflow enables a novel hardware design
approach unlocked by an emergent pruning scheme, periodic pattern-based
sparsity (PPS). By exploiting the regularity of PPS, our sparsity-aware
compiler optimally reorders the weights and uses a simple indexing unit in
hardware to create matches between the weights and activations. Through the
compiler-hardware codesign, SPS dataflow enjoys higher degrees of parallelism
while being free of the high indexing overhead and without model accuracy loss.
Evaluated on popular benchmarks such as VGG and ResNet, the SPS dataflow and
accompanying neural network compiler outperform prior work in convolutional
neural network (CNN) accelerator designs targeting FPGA devices. Against other
sparsity-supporting weight storage formats, SPS results in 4.49x energy
efficiency gain while lowering storage requirements by 3.67x for total weight
storage (non-pruned weights plus indexing) and 22,044x for indexing memory.
