  Tartan (TRT), a hardware accelerator for inference with Deep Neural Networks
(DNNs), is presented and evaluated on Convolutional Neural Networks. TRT
exploits the variable per layer precision requirements of DNNs to deliver
execution time that is proportional to the precision p in bits used per layer
for convolutional and fully-connected layers. Prior art has demonstrated an
accelerator with the same execution performance only for convolutional layers.
Experiments on image classification CNNs show that on average across all
networks studied, TRT outperforms a state-of-the-art bit-parallel accelerator
by 1:90x without any loss in accuracy while it is 1:17x more energy efficient.
TRT requires no network retraining while it enables trading off accuracy for
additional improvements in execution performance and energy efficiency. For
example, if a 1% relative loss in accuracy is acceptable, TRT is on average
2:04x faster and 1:25x more energy efficient than a conventional bit-parallel
accelerator. A Tartan configuration that processes 2-bits at time, requires
less area than the 1-bit configuration, improves efficiency to 1:24x over the
bit-parallel baseline while being 73% faster for convolutional layers and 60%
faster for fully-connected layers is also presented.
