  In recent years, deep neural networks (DNNs), have yielded strong results on
a wide range of applications. Graphics Processing Units (GPUs) have been one
key enabling factor leading to the current popularity of DNNs. However, despite
increasing hardware flexibility and software programming toolchain maturity,
high efficiency GPU programming remains difficult: it suffers from high
complexity, low productivity, and low portability. GPU vendors such as NVIDIA
have spent enormous effort to write special-purpose DNN libraries. However, on
other hardware targets, especially mobile GPUs, such vendor libraries are not
generally available. Thus, the development of portable, open, high-performance,
energy-efficient GPU code for DNN operations would enable broader deployment of
DNN-based algorithms. Toward this end, this work presents a framework to enable
productive, high-efficiency GPU programming for DNN computations across
hardware platforms and programming models. In particular, the framework
provides specific support for metaprogramming, autotuning, and DNN-tailored
data types. Using our framework, we explore implementing DNN operations on
three different hardware targets: NVIDIA, AMD, and Qualcomm GPUs. On NVIDIA
GPUs, we show both portability between OpenCL and CUDA as well competitive
performance compared to the vendor library. On Qualcomm GPUs, we show that our
framework enables productive development of target-specific optimizations, and
achieves reasonable absolute performance. Finally, On AMD GPUs, we show initial
results that indicate our framework can yield reasonable performance on a new
platform with minimal effort.
