  We consider the problem of optimal tracking control of unknown discrete-time
nonlinear nonzero-sum games. The related state-of-art literature is mostly
focused on Policy Iteration algorithms and multiple neural network
approximation, which may lead to practical implementation challenges and high
computational burden. To overcome these problems, we propose a novel
Q-function-based multi-step Value Iteration algorithm, which provides the
potential to accelerate convergence speed and improve the quality of solutions,
with an easy-to-realize initialization condition. A critic-only least squares
implementation approach is then employed, which alleviates the computational
complexity of commonly used multiple neural network-based methods. Afterwards,
by introducing the coupled Bellman operator, a novel linear programming
approach is derived, based on which Nash equilibria can be approximately
computed by solving a set of tractable finite-dimensional optimization
problems. We evaluate the tracking control capabilities of the proposed
algorithms to the problem of fully-automated dual-hormone (i.e., insulin and
glucagon) glucose control in Type 1 Diabetes Mellitus. The U.S. FDA-accepted
DMMS.R simulator from the Epsilon Group is used to conduct extensive in-silico
clinical studies on virtual patients under a variety of completely unannounced
meal and exercise scenarios. Simulation results demonstrate the high
reliability and exceptional performance of the proposed multi-step algorithmic
framework to critical complex systems.
