  In this paper, we study the problem of robust cooperative multi-agent
reinforcement learning (RL) where a large number of cooperative agents with
distributed information aim to learn policies in the presence of
\emph{stochastic} and \emph{non-stochastic} uncertainties whose distributions
are respectively known and unknown. Focusing on policy optimization that
accounts for both types of uncertainties, we formulate the problem in a
worst-case (minimax) framework, which is is intractable in general. Thus, we
focus on the Linear Quadratic setting to derive benchmark solutions. First,
since no standard theory exists for this problem due to the distributed
information structure, we utilize the Mean-Field Type Game (MFTG) paradigm to
establish guarantees on the solution quality in the sense of achieved Nash
equilibrium of the MFTG. This in turn allows us to compare the performance
against the corresponding original robust multi-agent control problem. Then, we
propose a Receding-horizon Gradient Descent Ascent RL algorithm to find the
MFTG Nash equilibrium and we prove a non-asymptotic rate of convergence.
Finally, we provide numerical experiments to demonstrate the efficacy of our
approach relative to a baseline algorithm.
