  As deep neural networks (DNNs) prove their importance and feasibility, more
and more DNN-based apps, such as detection and classification of objects, have
been developed and deployed on autonomous vehicles (AVs). To meet their growing
expectations and requirements, AVs should "optimize" use of their limited
onboard computing resources for multiple concurrent in-vehicle apps while
satisfying their timing requirements (especially for safety). That is,
real-time AV apps should share the limited on-board resources with other
concurrent apps without missing their deadlines dictated by the frame rate of a
camera that generates and provides input images to the apps. However, most, if
not all, of existing DNN solutions focus on enhancing the concurrency of their
specific hardware without dynamically optimizing/modifying the DNN apps'
resource requirements, subject to the number of running apps, owing to their
high computational cost. To mitigate this limitation, we propose DynaMIX
(Dynamic MIXed-precision model construction), which optimizes the resource
requirement of concurrent apps and aims to maximize execution accuracy. To
realize a real-time resource optimization, we formulate an optimization problem
using app performance profiles to consider both the accuracy and worst-case
latency of each app. We also propose dynamic model reconfiguration by lazy
loading only the selected layers at runtime to reduce the overhead of loading
the entire model. DynaMIX is evaluated in terms of constraint satisfaction and
inference accuracy for a multi-tasking system and compared against
state-of-the-art solutions, demonstrating its effectiveness and feasibility
under various environmental/operating conditions.
