  Neural Networks have become one of the most successful universal machine
learning algorithms. They play a key role in enabling machine vision and speech
recognition for example. Their computational complexity is enormous and comes
along with equally challenging memory requirements, which limits deployment in
particular within energy constrained, embedded environments. In order to
address these implementation challenges, a broad spectrum of new customized and
heterogeneous hardware architectures have emerged, often accompanied with
co-designed algorithms to extract maximum benefit out of the hardware.
Furthermore, numerous optimization techniques are being explored for neural
networks to reduce compute and memory requirements while maintaining accuracy.
This results in an abundance of algorithmic and architectural choices, some of
which fit specific use cases better than others.
  For system level designers, there is currently no good way to compare the
variety of hardware, algorithm and optimization options. While there are many
benchmarking efforts in this field, they cover only subsections of the embedded
design space. None of the existing benchmarks support essential algorithmic
optimizations such as quantization, an important technique to stay on chip, or
specialized heterogeneous hardware architectures. We propose a novel benchmark
suite, QuTiBench, that addresses this need. QuTiBench is a novel multi-tiered
benchmarking methodology that supports algorithmic optimizations such as
quantization and helps system developers understand the benefits and
limitations of these novel compute architectures in regard to specific neural
networks and will help drive future innovation. We invite the community to
contribute to QuTiBench in order to support the full spectrum of choices in
implementing machine learning systems.
