  Deep Neural Networks (DNNs) have transformed the field of machine learning
and are widely deployed in many applications involving image, video, speech and
natural language processing. The increasing compute demands of DNNs have been
widely addressed through Graphics Processing Units (GPUs) and specialized
accelerators. However, as model sizes grow, these von Neumann architectures
require very high memory bandwidth to keep the processing elements utilized as
a majority of the data resides in the main memory. Processing in memory has
been proposed as a promising solution for the memory wall bottleneck for ML
workloads. In this work, we propose a new DRAM-based processing-in-memory (PIM)
multiplication primitive coupled with intra-bank accumulation to accelerate
matrix vector operations in ML workloads. The proposed multiplication primitive
adds < 1% area overhead and does not require any change in the DRAM
peripherals. Therefore, the proposed multiplication can be easily adopted in
commodity DRAM chips. Subsequently, we design a DRAM-based PIM architecture,
data mapping scheme and dataflow for executing DNNs within DRAM. System
evaluations performed on networks like AlexNet, VGG16 and ResNet18 show that
the proposed architecture, mapping, and data flow can provide up to 19.5x
speedup over an NVIDIA Titan Xp GPU highlighting the need to overcome the
memory bottleneck in future generations of DNN hardware.
