  Neuromorphic Systems-on-Chip (NSoCs) are becoming heterogeneous by
integrating general-purpose processors (GPPs) and neural processing units
(NPUs) on the same SoC. For embedded systems, an NSoC may need to execute user
applications built using a variety of machine learning models. We propose a
real-time scheduler, called PRISM, which can schedule machine learning models
on a heterogeneous NSoC either individually or concurrently to improve their
system performance. PRISM consists of the following four key steps. First, it
constructs an interprocessor communication (IPC) graph of a machine learning
model from a mapping and a self-timed schedule. Second, it creates a
transaction order for the communication actors and embeds this order into the
IPC graph. Third, it schedules the graph on an NSoC by overlapping
communication with the computation. Finally, it uses a Hill Climbing heuristic
to explore the design space of mapping operations on GPPs and NPUs to improve
the performance. Unlike existing schedulers which use only the NPUs of an NSoC,
PRISM improves performance by enabling batch, pipeline, and operation
parallelism via exploiting a platform's heterogeneity. For use-cases with
concurrent applications, PRISM uses a heuristic resource sharing strategy and a
non-preemptive scheduling to reduce the expected wait time before concurrent
operations can be scheduled on contending resources. Our extensive evaluations
with 20 machine learning workloads show that PRISM significantly improves the
performance per watt for both individual applications and use-cases when
compared to state-of-the-art schedulers.
