  With the rapid development of deep learning models and hardware support for
dense computing, the deep learning workload characteristics changed
significantly from a few hot spots on compute-intensive operations to a broad
range of operations scattered across the models. Accelerating a few
compute-intensive operations using the expert-tuned implementation of
primitives does not fully exploit the performance potential of AI hardware.
Various efforts have been made to compile a full deep neural network (DNN)
graph. One of the biggest challenges is to achieve high-performance tensor
compilation by generating expert level performance code for the dense
compute-intensive operations and applying compilation optimization at the scope
of DNN computation graph across multiple compute-intensive operations.
  We present oneDNN Graph Compiler, a tensor compiler that employs a hybrid
approach of using techniques from both compiler optimization and expert-tuned
kernels for high performance code generation of the deep neural network graph.
oneDNN Graph Compiler addresses unique optimization challenges in the deep
learning domain, such as low-precision computation, aggressive fusion of graph
operations, optimization for static tensor shapes and memory layout, constant
weight optimization, and memory buffer reuse. Experimental results demonstrate
significant performance gains over existing tensor compiler and primitives
library for performance-critical DNN computation graphs and end-to-end models
on Intel Xeon Scalable Processors.
