  This paper investigates the optimal control problem for a class of
discrete-time stochastic systems subject to additive and multiplicative noises.
A stochastic Lyapunov equation and a stochastic algebra Riccati equation are
established for the existence of the optimal admissible control policy. A
model-free reinforcement learning algorithm is proposed to learn the optimal
admissible control policy using the data of the system states and inputs
without requiring any knowledge of the system matrices. It is proven that the
learning algorithm converges to the optimal admissible control policy. The
implementation of the model-free algorithm is based on batch least squares and
numerical average. The proposed algorithm is illustrated through a numerical
example, which shows our algorithm outperforms other policy iteration
algorithms.
