  This paper examines the design space trade-offs of DNNs accelerators aiming
to achieve competitive performance and efficiency metrics for all four
combinations of dense or sparse activation/weight tensors. To do so, we
systematically examine the overheads of supporting sparsity on top of an
optimized dense core. These overheads are modeled based on parameters that
indicate how a multiplier can borrow a nonzero operation from the neighboring
multipliers or future cycles. As a result of this exploration, we identify a
few promising designs that perform better than prior work. Our findings suggest
that even the best design targeting dual sparsity yields a 20%-30% drop in
power efficiency when performing on single sparse models, i.e., those with only
sparse weight or sparse activation tensors. We found that one can reuse
resources of the same core to maintain high performance and efficiency when
running single sparsity or dense models. We call this hybrid architecture
Griffin. Griffin is 1.2, 3.0, 3.1, and 1.4X more power-efficient than
state-of-the-art sparse architectures, for dense, weight-only sparse,
activation-only sparse, and dual sparse models, respectively.
