  As AI-based applications become pervasive, CPU vendors are starting to
incorporate matrix engines within the datapath to boost efficiency. Systolic
arrays have been the premier architectural choice as matrix engines in offload
accelerators. However, we demonstrate that incorporating them inside CPUs can
introduce under-utilization and stalls due to limited register storage to
amortize the fill and drain times of the array. To address this, we propose
RASA, Register-Aware Systolic Array. We develop techniques to divide an
execution stage into several sub-stages and overlap instructions to hide
overheads and run them concurrently. RASA-based designs improve performance
significantly with negligible area and power overhead.
