  Training AI with strong and rich strategies in multi-agent environments
remains an important research topic in Deep Reinforcement Learning (DRL). The
AI's strength is closely related to its diversity of strategies, and this
relationship can guide us to train AI with both strong and rich strategies. To
prove this point, we propose Diversity is Strength (DIS), a novel DRL training
framework that can simultaneously train multiple kinds of AIs. These AIs are
linked through an interconnected history model pool structure, which enhances
their capabilities and strategy diversities. We also design a model evaluation
and screening scheme to select the best models to enrich the model pool and
obtain the final AI. The proposed training method provides diverse,
generalizable, and strong AI strategies without using human data. We tested our
method in an AI competition based on Google Research Football (GRF) and won the
5v5 and 11v11 tracks. The method enables a GRF AI to have a high level on both
5v5 and 11v11 tracks for the first time, which are under complex multi-agent
environments. The behavior analysis shows that the trained AI has rich
strategies, and the ablation experiments proved that the designed modules
benefit the training process.
