  We consider a joint uplink and downlink scheduling problem of a fully
distributed wireless networked control system (WNCS) with a limited number of
frequency channels. Using elements of stochastic systems theory, we derive a
sufficient stability condition of the WNCS, which is stated in terms of both
the control and communication system parameters. Once the condition is
satisfied, there exists a stationary and deterministic scheduling policy that
can stabilize all plants of the WNCS. By analyzing and representing the
per-step cost function of the WNCS in terms of a finite-length countable vector
state, we formulate the optimal transmission scheduling problem into a Markov
decision process and develop a deep reinforcement learning (DRL) based
framework for solving it. To tackle the challenges of a large action space in
DRL, we propose novel action space reduction and action embedding methods for
the DRL framework that can be applied to various algorithms, including Deep
Q-Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed
Deep Deterministic Policy Gradient (TD3). Numerical results show that the
proposed algorithm significantly outperforms benchmark policies.
