  Most deep neural networks deployed today are trained using GPUs via
high-level frameworks such as TensorFlow and PyTorch. This paper describes
changes we made to the GPGPU-Sim simulator to enable it to run PyTorch by
running PTX kernels included in NVIDIA's cuDNN library. We use the resulting
modified simulator, which has been made available publicly with this paper, to
study some simple deep learning workloads. With our changes to GPGPU-Sim's
functional simulation model, we find GPGPU-Sim performance model running a
cuDNN enabled implementation of LeNet for MNIST reports results within 30% of
real hardware. Using GPGPU-Sim's AerialVision performance analysis tool we
observe that cuDNN API calls contain many varying phases and appear to include
potentially inefficient microarchitecture behaviour such as DRAM partition bank
camping, at least when executed on GPGPU-Sim's current performance model.
