  With the increase in the computation intensity of the chip, the mismatch
between computation layer shapes and the available computation resource
significantly limits the utilization of the chip. Driven by this observation,
prior works discuss spatial accelerators or dataflow architecture to maximize
the throughput. However, using spatial accelerators could potentially increase
the execution latency. In this work, we first systematically investigate two
execution models: (1) sequentially (temporally) launch one monolithic
accelerator, and (2) spatially launch multiple accelerators. From the
observations, we find that there is a latency throughput tradeoff between these
two execution models, and combining these two strategies together can give us a
more efficient latency throughput Pareto front. To achieve this, we propose
spatial sequential architecture (SSR) and SSR design automation framework to
explore both strategies together when deploying deep learning inference. We use
the 7nm AMD Versal ACAP VCK190 board to implement SSR accelerators for four
end-to-end transformer-based deep learning models. SSR achieves average
throughput gains of 2.53x, 35.71x, and 14.20x under different batch sizes
compared to the 8nm Nvidia GPU A10G, 16nm AMD FPGAs ZCU102, and U250. The
average energy efficiency gains are 8.51x, 6.75x, and 21.22x, respectively.
Compared with the sequential-only solution and spatial-only solution on VCK190,
our spatial-sequential-hybrid solutions achieve higher throughput under the
same latency requirement and lower latency under the same throughput
requirement. We also use SSR analytical models to demonstrate how to use SSR to
optimize solutions on other computing platforms, e.g., 14nm Intel Stratix 10
NX.
