  Remote state estimation of large-scale distributed dynamic processes plays an
important role in Industry 4.0 applications. In this paper, we focus on the
transmission scheduling problem of a remote estimation system. First, we derive
some structural properties of the optimal sensor scheduling policy over fading
channels. Then, building on these theoretical guidelines, we develop a
structure-enhanced deep reinforcement learning (DRL) framework for optimal
scheduling of the system to achieve the minimum overall estimation mean-square
error (MSE). In particular, we propose a structure-enhanced action selection
method, which tends to select actions that obey the policy structure. This
explores the action space more effectively and enhances the learning efficiency
of DRL agents. Furthermore, we introduce a structure-enhanced loss function to
add penalties to actions that do not follow the policy structure. The new loss
function guides the DRL to converge to the optimal policy structure quickly.
Our numerical experiments illustrate that the proposed structure-enhanced DRL
algorithms can save the training time by 50% and reduce the remote estimation
MSE by 10% to 25% when compared to benchmark DRL algorithms. In addition, we
show that the derived structural properties exist in a wide range of dynamic
scheduling problems that go beyond remote state estimation.
