  Recent years have witnessed great progress in deep neural networks for
real-time applications. However, most existing works do not explicitly consider
the general case where the device's state and the available resources fluctuate
over time, and none of them investigate or address the impact of varying
computational resources for online video understanding tasks. This paper
proposes a System-status-aware Adaptive Network (SAN) that considers the
device's real-time state to provide high-quality predictions with low delay.
Usage of our agent's policy improves efficiency and robustness to fluctuations
of the system status. On two widely used video understanding tasks, SAN obtains
state-of-the-art performance while constantly keeping processing delays low.
Moreover, training such an agent on various types of hardware configurations is
not easy as the labeled training data might not be available, or can be
computationally prohibitive. To address this challenging problem, we propose a
Meta Self-supervised Adaptation (MSA) method that adapts the agent's policy to
new hardware configurations at test-time, allowing for easy deployment of the
model onto other unseen hardware platforms.
