  The acceleration of deep-learning kernels in hardware relies on matrix
multiplications that are executed efficiently on Systolic Arrays (SA). To
effectively trade off deep-learning training/inference quality with hardware
cost, SA accelerators employ reduced-precision Floating-Point (FP) arithmetic.
In this work, we demonstrate the need for new pipeline organizations to reduce
latency and improve energy efficiency of reduced-precision FP operators for the
chained multiply-add operation imposed by the structure of the SA. The proposed
skewed pipeline design reorganizes the pipelined operation of the FP
multiply-add units to enable new forwarding paths for the exponent logic, which
allow for parallel execution of the pipeline stages of consecutive PEs. As a
result, the latency of the matrix multiplication operation within the SA is
significantly reduced with minimal hardware cost, thereby yielding an energy
reduction of 8% and 11% for the examined state-of-the-art CNNs.
