  This research work proposes a design of an analog ReRAM-based PIM
(processing-in-memory) architecture for fast and efficient CNN (convolutional
neural network) inference. For the overall architecture, we use the basic
hardware hierarchy such as node, tile, core, and subarray. On the top of that,
we design intra-layer pipelining, inter-layer pipelining, and batch pipelining
to exploit parallelism in the architecture and increase overall throughput for
the inference of an input image stream. We also optimize the performance of the
NoC (network-on-chip) routers by decreasing hop counts using SMART
(single-cycle multi-hop asynchronous repeated traversal) flow control. Finally,
we experiment with weight replications for different CNN layers in VGG (A-E)
for large-scale data set ImageNet. In our simulation, we achieve 40.4027 TOPS
(tera-operations per second) for the best-case performance, which corresponds
to over 1029 FPS (frames per second). We also achieve 3.5914 TOPS/W
(tera-operaions per second per watt) for the best-case energy efficiency. In
addition, the architecture with aggressive pipelining and weight replications
can achieve 14X speedup compared to the baseline architecture with basic
pipelining, and SMART flow control achieves 1.08X speedup in this architecture
compared to the baseline. Last but not least, we also evaluate the performance
of SMART flow control using synthetic traffic.
