  In this paper, we propose a design of a model-free networked controller for a
nonlinear plant whose mathematical model is unknown. In a networked control
system, the controller and plant are located away from each other and exchange
data over a network, which causes network delays that may fluctuate randomly
due to network routing. So, in this paper, we assume that the current network
delay is not known but the maximum value of fluctuating network delays is known
beforehand. Moreover, we also assume that the sensor cannot observe all state
variables of the plant. Under these assumption, we apply continuous deep
Q-learning to the design of the networked controller. Then, we introduce an
extended state consisting of a sequence of past control inputs and outputs as
inputs to the deep neural network. By simulation, it is shown that, using the
extended state, the controller can learn a control policy robust to the
fluctuation of the network delays under the partial observation.
