  Convolutional Neural Networks have dramatically improved in recent years,
surpassing human accuracy on certain problems and performance exceeding that of
traditional computer vision algorithms. While the compute pattern in itself is
relatively simple, significant compute and memory challenges remain as CNNs may
contain millions of floating-point parameters and require billions of
floating-point operations to process a single image. These computational
requirements, combined with storage footprints that exceed typical cache sizes,
pose a significant performance and power challenge for modern compute
architectures. One of the promising opportunities to scale performance and
power efficiency is leveraging reduced precision representations for all
activations and weights as this allows to scale compute capabilities, reduce
weight and feature map buffering requirements as well as energy consumption.
While a small reduction in accuracy is encountered, these Quantized Neural
Networks have been shown to achieve state-of-the-art accuracy on standard
benchmark datasets, such as MNIST, CIFAR-10, SVHN and even ImageNet, and thus
provide highly attractive design trade-offs. Current research has focused
mainly on the implementation of extreme variants with full binarization of
weights and or activations, as well typically smaller input images. Within this
paper, we investigate the scalability of dataflow architectures with respect to
supporting various precisions for both weights and activations, larger image
dimensions, and increasing numbers of feature map channels. Key contributions
are a formalized approach to understanding the scalability of the existing
hardware architecture with cost models and a performance prediction as a
function of the target device size. We provide validating experimental results
for an ImageNet classification on a server-class platform, namely the AWS F1
node.
