  The choice of convolutional routines (primitives) to implement neural
networks has a tremendous impact on their inference performance (execution
speed) on a given hardware platform. To optimise a neural network by primitive
selection, the optimal primitive is identified for each layer of the network.
This process requires a lengthy profiling stage, iterating over all the
available primitives for each layer configuration, to measure their execution
time on the target platform. Because each primitive exploits the hardware in
different ways, new profiling is needed to obtain the best performance when
moving to another platform. In this work, we propose to replace this
prohibitively expensive profiling stage with a machine learning based approach
of performance modeling. Our approach speeds up the optimisation time
drastically. After training, our performance model can estimate the performance
of convolutional primitives in any layer configuration. The time to optimise
the execution of large neural networks via primitive selection is reduced from
hours to just seconds. Our performance model is easily transferable to other
target platforms. We demonstrate this by training a performance model on an
Intel platform and performing transfer learning to AMD and ARM processor
devices with minimal profiled samples.
