  As spiking-based deep learning inference applications are increasing in
embedded systems, these systems tend to integrate neuromorphic accelerators
such as $\mu$Brain to improve energy efficiency. We propose a $\mu$Brain-based
scalable many-core neuromorphic hardware design to accelerate the computations
of spiking deep convolutional neural networks (SDCNNs). To increase energy
efficiency, cores are designed to be heterogeneous in terms of their neuron and
synapse capacity (big cores have higher capacity than the little ones), and
they are interconnected using a parallel segmented bus interconnect, which
leads to lower latency and energy compared to a traditional mesh-based
Network-on-Chip (NoC). We propose a system software framework called SentryOS
to map SDCNN inference applications to the proposed design. SentryOS consists
of a compiler and a run-time manager. The compiler compiles an SDCNN
application into subnetworks by exploiting the internal architecture of big and
little $\mu$Brain cores. The run-time manager schedules these sub-networks onto
cores and pipeline their execution to improve throughput. We evaluate the
proposed big little many-core neuromorphic design and the system software
framework with five commonlyused SDCNN inference applications and show that the
proposed solution reduces energy (between 37% and 98%), reduces latency
(between 9% and 25%), and increases application throughput (between 20% and
36%). We also show that SentryOS can be easily extended for other spiking
neuromorphic accelerators.
