  We study performance characteristics of convolutional neural networks (CNN)
for mobile computer vision systems. CNNs have proven to be a powerful and
efficient approach to implement such systems. However, the system performance
depends largely on the utilization of hardware accelerators, which are able to
speed up the execution of the underlying mathematical operations tremendously
through massive parallelism. Our contribution is performance characterization
of multiple CNN-based models for object recognition and detection with several
different hardware platforms and software frameworks, using both local
(on-device) and remote (network-side server) computation. The measurements are
conducted using real workloads and real processing platforms. On the platform
side, we concentrate especially on TensorFlow and TensorRT. Our measurements
include embedded processors found on mobile devices and high-performance
processors that can be used on the network side of mobile systems. We show that
there exists significant latency--throughput trade-offs but the behavior is
very complex. We demonstrate and discuss several factors that affect the
performance and yield this complex behavior.
