  Dense Matrix Multiplication (MatMul) is arguably one of the most ubiquitous
compute-intensive kernels, spanning linear algebra, DSP, graphics, and machine
learning applications. Thus, MatMul optimization is crucial not only in
high-performance processors but also in embedded low-power platforms. Several
Instruction Set Architectures (ISAs) have recently included matrix extensions
to improve MatMul performance and efficiency at the cost of added matrix
register files and units. In this paper, we propose Matrix eXtension (MX), a
lightweight approach that builds upon the open-source RISC-V Vector (RVV) ISA
to boost MatMul energy efficiency. Instead of adding expensive dedicated
hardware, MX uses the pre-existing vector register file and functional units to
create a hybrid vector/matrix engine at a negligible area cost (< 3%), which
comes from a compact near-FPU tile buffer for higher data reuse, and no clock
frequency overhead. We implement MX on a compact and highly energy-optimized
RVV processor and evaluate it in both a Dual- and 64-Core cluster in a 12-nm
technology node. MX boosts the Dual-Core's energy efficiency by 10% for a
double-precision 64x64x64 matrix multiplication with the same FPU utilization
(~97%) and by 25% on the 64-Core cluster for the same benchmark on 32-bit data,
with a 56% performance gain.
