  In this work, we present a novel scheduling framework enabling anytime
perception for deep neural network (DNN) based 3D object detection pipelines.
We focus on computationally expensive region proposal network (RPN) and
per-category multi-head detector components, which are common in 3D object
detection pipelines, and make them deadline-aware. We propose a scheduling
algorithm, which intelligently selects the subset of the components to make
effective time and accuracy trade-off on the fly. We minimize accuracy loss of
skipping some of the neural network sub-components by projecting previously
detected objects onto the current scene through estimations. We apply our
approach to a state-of-art 3D object detection network, PointPillars, and
evaluate its performance on Jetson Xavier AGX using nuScenes dataset. Compared
to the baselines, our approach significantly improve the network's accuracy
under various deadline constraints.
