  In this paper, a novel decentralized intelligent adaptive optimal strategy
has been developed to solve the pursuit-evasion game for massive Multi-Agent
Systems (MAS) under uncertain environment. Existing strategies for
pursuit-evasion games are neither efficient nor practical for large population
multi-agent system due to the notorious "Curse of dimensionality" and
communication limit while the agent population is large. To overcome these
challenges, the emerging mean field game theory is adopted and further
integrated with reinforcement learning to develop a novel decentralized
intelligent adaptive strategy with a new type of adaptive dynamic programing
architecture named the Actor-Critic-Mass (ACM). Through online approximating
the solution of the coupled mean field equations, the developed strategy can
obtain the optimal pursuit-evasion policy even for massive MAS under uncertain
environment. In the proposed ACM learning based strategy, each agent maintains
five neural networks, which are 1) the critic neural network to approximate the
solution of the HJI equation for each individual agent; 2) the mass neural
network to estimate the population density function (i.e., mass) of the group;
3) the actor neural network to approximate the decentralized optimal strategy,
and 4) two more neural networks are designed to estimate the opponents' group
mass as well as the optimal cost function. Eventually, a comprehensive
numerical simulation has been provided to demonstrate the effectiveness of the
designed strategy.
