  During the past few years, interest in convolutional neural networks (CNNs)
has risen constantly, thanks to their excellent performance on a wide range of
recognition and classification tasks. However, they suffer from the high level
of complexity imposed by the high-dimensional convolutions in convolutional
layers. Within scenarios with limited hardware resources and tight power and
latency constraints, the high computational complexity of CNNs makes them
difficult to be exploited. Hardware solutions have striven to reduce the power
consumption using low-power techniques, and to limit the processing time by
increasing the number of processing elements (PEs). While most of ASIC designs
claim a peak performance of a few hundred giga operations per seconds, their
average performance is substantially lower when applied to state-of-the-art
CNNs such as AlexNet, VGGNet and ResNet, leading to low resource utilization.
Their performance efficiency is limited to less than 55% on average, which
leads to unnecessarily high processing latency and silicon area. In this paper,
we propose a dataflow which enables to perform both the fully-connected and
convolutional computations for any filter/layer size using the same PEs. We
then introduce a multi-mode inference engine (MMIE) based on the proposed
dataflow. Finally, we show that the proposed MMIE achieves a performance
efficiency of more than 84% when performing the computations of the three
renown CNNs (i.e., AlexNet, VGGNet and ResNet), outperforming the best
architecture in the state-of-the-art in terms of energy consumption, processing
latency and silicon area.
