  With the success of deep learning, object recognition systems that can be
deployed for real-world applications are becoming commonplace. However,
inference that needs to largely take place on the `edge' (not processed on
servers), is a highly computational and memory intensive workload, making it
intractable for low-power mobile nodes and remote security applications. To
address this challenge, this paper proposes a low-power (5W) end-to-end
neuromorphic framework for object tracking and classification using event-based
cameras that possess desirable properties such as low power consumption (5-14
mW) and high dynamic range (120 dB). Nonetheless, unlike traditional approaches
of using event-by-event processing, this work uses a mixed frame and event
approach to get energy savings with high performance. Using a frame-based
region proposal method based on the density of foreground events, a
hardware-friendly object tracking is implemented using the apparent object
velocity while tackling occlusion scenarios. For low-power classification of
the tracked objects, the event camera is interfaced to IBM TrueNorth, which is
time-multiplexed to tackle up to eight instances for a traffic monitoring
application. The frame-based object track input is converted back to spikes for
Truenorth classification via the energy efficient deep network (EEDN) pipeline.
Using originally collected datasets, we train the TrueNorth model on the
hardware track outputs, instead of using ground truth object locations as
commonly done, and demonstrate the efficacy of our system to handle practical
surveillance scenarios. Finally, we compare the proposed methodologies to
state-of-the-art event-based systems for object tracking and classification,
and demonstrate the use case of our neuromorphic approach for low-power
applications without sacrificing on performance.
