  Deep learning based recommendation systems form the backbone of most
personalized cloud services. Though the computer architecture community has
recently started to take notice of deep recommendation inference, the resulting
solutions have taken wildly different approaches - ranging from near memory
processing to at-scale optimizations. To better design future hardware systems
for deep recommendation inference, we must first systematically examine and
characterize the underlying systems-level impact of design decisions across the
different levels of the execution stack. In this paper, we characterize eight
industry-representative deep recommendation models at three different levels of
the execution stack: algorithms and software, systems platforms, and hardware
microarchitectures. Through this cross-stack characterization, we first show
that system deployment choices (i.e., CPUs or GPUs, batch size granularity) can
give us up to 15x speedup. To better understand the bottlenecks for further
optimization, we look at both software operator usage breakdown and CPU
frontend and backend microarchitectural inefficiencies. Finally, we model the
correlation between key algorithmic model architecture features and hardware
bottlenecks, revealing the absence of a single dominant algorithmic component
behind each hardware bottleneck.
