  The recent research advances in deep learning have led to the development of
small and powerful Convolutional Neural Network (CNN) architectures. Meanwhile
Field Programmable Gate Arrays (FPGAs) has become a popular hardware target
choice for their deployment, splitting into two main implementation categories:
streaming hardware architectures and single computation engine design
approaches. The streaming hardware architectures generally require implementing
every layer as a discrete processing unit, and are suitable for smaller
software models that could fit in their unfolded versions into
resource-constrained targets. On the other hand, single computation engines can
be scaled to fit into a device to execute CNN models of different sizes and
complexities, however, the achievable performance of one-size-fits-all
implementations may vary across CNNs with different workload attributes leading
to inefficient utilization of hardware resources. By combing the advantages of
both of the above methods, this work proposes a new design paradigm called
semi-streaming architecture, where layerspecialized configurable engines are
used for network realization. As a proof of concept this paper presents a set
of five layerspecialized configurable processing engines for implementing 8-bit
quantized MobilenevV2 CNN model. The engines are chained to partially preserve
data streaming and tuned individually to efficiently process specific types of
layers: normalized addition of residuals, depthwise, pointwise (expansion and
projection), and standard 2D convolution layers capable of delivering 5.4GOp/s,
16GOp/s, 27.2GOp/s, 27.2GOp/s and 89.6GOp/s, respectively, with the overall
energy efficiency of 5.32GOp/s/W at a 100MHz system clock, requiring total
power of 6.2W on a XCZU7EV SoC FPGA.
