  Real-time perception requires planned resource utilization. Computational
planning in real-time perception is governed by two considerations -- accuracy
and latency. There exist run-time decisions (e.g. choice of input resolution)
that induce tradeoffs affecting performance on a given hardware, arising from
intrinsic (content, e.g. scene clutter) and extrinsic (system, e.g. resource
contention) characteristics.
  Earlier runtime execution frameworks employed rule-based decision algorithms
and operated with a fixed algorithm latency budget to balance these concerns,
which is sub-optimal and inflexible. We propose Chanakya, a learned approximate
execution framework that naturally derives from the streaming perception
paradigm, to automatically learn decisions induced by these tradeoffs instead.
Chanakya is trained via novel rewards balancing accuracy and latency
implicitly, without approximating either objectives. Chanakya simultaneously
considers intrinsic and extrinsic context, and predicts decisions in a flexible
manner. Chanakya, designed with low overhead in mind, outperforms
state-of-the-art static and dynamic execution policies on public datasets on
both server GPUs and edge devices.
