  The ever-increasing size and computational complexity of today's
machine-learning algorithms pose an increasing strain on the underlying
hardware. In this light, novel and dedicated architectural solutions are
required to optimize energy efficiency by leveraging opportunities (such as
intrinsic parallelism and robustness to quantization errors) exposed by
algorithms. We herein address this challenge by introducing a flexible
two-stages computing pipeline. The pipeline can support fine-grained operand
quantization through software-supported Single Instruction Multiple Data (SIMD)
operations. Moreover, it can efficiently execute sequential multiplications
over SIMD sub-words thanks to zero-skipping and Canonical Signed Digit (CSD)
coding. Finally, a lightweight repacking unit allows changing the bitwidth of
sub-words at run-time dynamically. These features are implemented within a
tight energy and area budget. Indeed, experimental results showcase that our
approach greatly outperforms traditional hardware SIMD ones both in terms of
area and energy requirements. In particular, our pipeline occupies up to 53.1%
smaller than a hardware SIMD one supporting the same sub-word widths, while
performing multiplication up to 88.8% more efficiently.
