  Wireless networked control system (WNCS) connecting sensors, controllers, and
actuators via wireless communications is a key enabling technology for highly
scalable and low-cost deployment of control systems in the Industry 4.0 era.
Despite the tight interaction of control and communications in WNCSs, most
existing works adopt separative design approaches. This is mainly because the
co-design of control-communication policies requires large and hybrid state and
action spaces, making the optimal problem mathematically intractable and
difficult to be solved effectively by classic algorithms. In this paper, we
systematically investigate deep learning (DL)-based estimator-control-scheduler
co-design for a model-unknown nonlinear WNCS over wireless fading channels. In
particular, we propose a co-design framework with the awareness of the sensor's
age-of-information (AoI) states and dynamic channel states. We propose a novel
deep reinforcement learning (DRL)-based algorithm for controller and scheduler
optimization utilizing both model-free and model-based data. An AoI-based
importance sampling algorithm that takes into account the data accuracy is
proposed for enhancing learning efficiency. We also develop novel schemes for
enhancing the stability of joint training. Extensive experiments demonstrate
that the proposed joint training algorithm can effectively solve the
estimation-control-scheduling co-design problem in various scenarios and
provide significant performance gain compared to separative design and some
benchmark policies.
