  We propose nnstreamer, a software system that handles neural networks as
filters of stream pipelines, applying the stream processing paradigm to neural
network applications. A new trend with the wide-spread of deep neural network
applications is on-device AI; i.e., processing neural networks directly on
mobile devices or edge/IoT devices instead of cloud servers. Emerging privacy
issues, data transmission costs, and operational costs signifies the need for
on-device AI especially when a huge number of devices with real-time data
processing are deployed. Nnstreamer efficiently handles neural networks with
complex data stream pipelines on devices, improving the overall performance
significantly with minimal efforts. Besides, nnstreamer simplifies the neural
network pipeline implementations and allows reusing off-shelf multimedia stream
filters directly; thus it reduces the developmental costs significantly.
Nnstreamer is already being deployed with a product releasing soon and is open
source software applicable to a wide range of hardware architectures and
software platforms.
