  The computation and memory-intensive nature of DNNs limits their use in many
mobile and embedded contexts. Application-specific integrated circuit (ASIC)
hardware accelerators employ matrix multiplication units (such as the systolic
arrays) and dedicated nonlinear function units to speed up DNN computations. A
close examination of these ASIC accelerators reveals that the designs are often
specialized and lack versatility across different networks, especially when the
networks have different types of computation. In this paper, we introduce a
novel systolic array architecture, which is capable of executing nonlinear
functions. By encompassing both inherent linear and newly enabled nonlinear
functions within the systolic arrays, the proposed architecture facilitates
versatile network inferences, substantially enhancing computational power and
energy efficiency. Experimental results show that employing this systolic array
enables seamless execution of entire DNNs, incurring only a negligible loss in
the network inference accuracy. Furthermore, assessment and evaluation with
FPGAs reveal that integrating nonlinear computation capacity into a systolic
array does not introduce extra notable (less than 1.5%) block memory memories
(BRAMs), look-up-tables (LUTs), or digital signal processors (DSPs) but a mere
13.3% - 24.1% more flip flops (FFs). In comparison to existing methodologies,
executing the networks with the proposed systolic array, which enables the
flexibility of different network models, yields up to 25.73x, 5.21x, and 1.54x
computational efficiency when compared to general-purpose CPUs, GPUs, and SoCs
respectively, while achieving comparable (83.4% - 135.8%) performance with the
conventional accelerators which are designed for specific neural network
models.
