  Collaboration among industrial Internet of Things (IoT) devices and edge
networks is essential to support computation-intensive deep neural network
(DNN) inference services which require low delay and high accuracy. Sampling
rate adaption which dynamically configures the sampling rates of industrial IoT
devices according to network conditions, is the key in minimizing the service
delay. In this paper, we investigate the collaborative DNN inference problem in
industrial IoT networks. To capture the channel variation and task arrival
randomness, we formulate the problem as a constrained Markov decision process
(CMDP). Specifically, sampling rate adaption, inference task offloading and
edge computing resource allocation are jointly considered to minimize the
average service delay while guaranteeing the long-term accuracy requirements of
different inference services. Since CMDP cannot be directly solved by general
reinforcement learning (RL) algorithms due to the intractable long-term
constraints, we first transform the CMDP into an MDP by leveraging the Lyapunov
optimization technique. Then, a deep RL-based algorithm is proposed to solve
the MDP. To expedite the training process, an optimization subroutine is
embedded in the proposed algorithm to directly obtain the optimal edge
computing resource allocation. Extensive simulation results are provided to
demonstrate that the proposed RL-based algorithm can significantly reduce the
average service delay while preserving long-term inference accuracy with a high
probability.
