  Convolutional Neural Networks (CNNs) have shown a great deal of success in
diverse application domains including computer vision, speech recognition, and
natural language processing. However, as the size of datasets and the depth of
neural network architectures continue to grow, it is imperative to design
high-performance and energy-efficient computing hardware for training CNNs. In
this paper, we consider the problem of designing specialized CPU-GPU based
heterogeneous manycore systems for energy-efficient training of CNNs. It has
already been shown that the typical on-chip communication infrastructures
employed in conventional CPU-GPU based heterogeneous manycore platforms are
unable to handle both CPU and GPU communication requirements efficiently. To
address this issue, we first analyze the on-chip traffic patterns that arise
from the computational processes associated with training two deep CNN
architectures, namely, LeNet and CDBNet, to perform image classification. By
leveraging this knowledge, we design a hybrid Network-on-Chip (NoC)
architecture, which consists of both wireline and wireless links, to improve
the performance of CPU-GPU based heterogeneous manycore platforms running the
above-mentioned CNN training workloads. The proposed NoC achieves 1.8x
reduction in network latency and improves the network throughput by a factor of
2.2 for training CNNs, when compared to a highly-optimized wireline mesh NoC.
For the considered CNN workloads, these network-level improvements translate
into 25% savings in full-system energy-delay-product (EDP). This demonstrates
that the proposed hybrid NoC for heterogeneous manycore architectures is
capable of significantly accelerating training of CNNs while remaining
energy-efficient.
