  Realizing today's cloud-level artificial intelligence functionalities
directly on devices distributed at the edge of the internet calls for edge
hardware capable of processing multiple modalities of sensory data (e.g. video,
audio) at unprecedented energy-efficiency. AI hardware architectures today
cannot meet the demand due to a fundamental "memory wall": data movement
between separate compute and memory units consumes large energy and incurs long
latency. Resistive random-access memory (RRAM) based compute-in-memory (CIM)
architectures promise to bring orders of magnitude energy-efficiency
improvement by performing computation directly within memory. However,
conventional approaches to CIM hardware design limit its functional flexibility
necessary for processing diverse AI workloads, and must overcome hardware
imperfections that degrade inference accuracy. Such trade-offs between
efficiency, versatility and accuracy cannot be addressed by isolated
improvements on any single level of the design. By co-optimizing across all
hierarchies of the design from algorithms and architecture to circuits and
devices, we present NeuRRAM - the first multimodal edge AI chip using RRAM CIM
to simultaneously deliver a high degree of versatility for diverse model
architectures, record energy-efficiency $5\times$ - $8\times$ better than prior
art across various computational bit-precisions, and inference accuracy
comparable to software models with 4-bit weights on all measured standard AI
benchmarks including accuracy of 99.0% on MNIST and 85.7% on CIFAR-10 image
classification, 84.7% accuracy on Google speech command recognition, and a 70%
reduction in image reconstruction error on a Bayesian image recovery task. This
work paves a way towards building highly efficient and reconfigurable edge AI
hardware platforms for the more demanding and heterogeneous AI applications of
the future.
