  Deep Neural Networks are becoming increasingly popular in always-on IoT edge
devices performing data analytics right at the source, reducing latency as well
as energy consumption for data communication. This paper presents CMSIS-NN,
efficient kernels developed to maximize the performance and minimize the memory
footprint of neural network (NN) applications on Arm Cortex-M processors
targeted for intelligent IoT edge devices. Neural network inference based on
CMSIS-NN kernels achieves 4.6X improvement in runtime/throughput and 4.9X
improvement in energy efficiency.
