  Remote state estimation of large-scale distributed dynamic processes plays an
important role in Industry 4.0 applications. In this paper, by leveraging the
theoretical results of structural properties of optimal scheduling policies, we
develop a structure-enhanced deep reinforcement learning (DRL) framework for
optimal scheduling of a multi-sensor remote estimation system to achieve the
minimum overall estimation mean-square error (MSE). In particular, we propose a
structure-enhanced action selection method, which tends to select actions that
obey the policy structure. This explores the action space more effectively and
enhances the learning efficiency of DRL agents. Furthermore, we introduce a
structure-enhanced loss function to add penalty to actions that do not follow
the policy structure. The new loss function guides the DRL to converge to the
optimal policy structure quickly. Our numerical results show that the proposed
structure-enhanced DRL algorithms can save the training time by 50% and reduce
the remote estimation MSE by 10% to 25%, when compared to benchmark DRL
algorithms.
