  There is a growing demand to deploy computation-intensive deep learning (DL)
models on resource-constrained mobile devices for real-time intelligent
applications. Equipped with a variety of processing units such as CPUs, GPUs,
and NPUs, the mobile devices hold potential to accelerate DL inference via
parallel execution across heterogeneous processors. Various efficient parallel
methods have been explored to optimize computation distribution, achieve load
balance, and minimize communication cost across processors. Yet their practical
effectiveness in the dynamic and diverse real-world mobile environment is less
explored. This paper presents a holistic empirical study to assess the
capabilities and challenges associated with parallel DL inference on
heterogeneous mobile processors. Through carefully designed experiments
covering various DL models, mobile software/hardware environments, workload
patterns, and resource availability, we identify limitations of existing
techniques and highlight opportunities for cross-level optimization.
