  Modern deep learning models have high memory and computation cost. To make
them fast and memory-cost efficient, structured model pruning is commonly used.
We find that pruning a model using a common training accelerator with large
systolic arrays is extremely performance-inefficient. To make a systolic array
efficient for pruning and training, we propose FlexSA, a flexible systolic
array architecture. FlexSA dynamically reconfigures the systolic array
structure and offers multiple sub-systolic operating modes, which are designed
for energy- and memory bandwidth-efficient processing of tensors with different
sizes and shapes. We also present a compilation heuristic for tiling
matrix-multiplication-and-accumulation operations in a training workload to
best utilize the resources of FlexSA. Based on our evaluation, FlexSA with the
proposed compilation heuristic improves compute resource utilization of pruning
and training modern CNN models by 37% compared to a conventional training
accelerator with a large systolic array. FlexSA also improves on-chip data
reuse by 1.7X saving 28% energy compared to naive systolic array splitting.
