  Communication-based multi-agent reinforcement learning (MARL) provides
information exchange between agents, which promotes the cooperation. However,
existing methods cannot perform well in the large-scale multi-agent system. In
this paper, we adopt neighboring communication and propose a Neighboring
Variational Information Flow (NVIF) to provide efficient communication for
agents. It employs variational auto-encoder to compress the shared information
into a latent state. This communication protocol does not rely dependently on a
specific task, so that it can be pre-trained to stabilize the MARL training.
Besides. we combine NVIF with Proximal Policy Optimization (NVIF-PPO) and Deep
Q Network (NVIF-DQN), and present a theoretical analysis to illustrate NVIF-PPO
can promote cooperation. We evaluate the NVIF-PPO and NVIF-DQN on MAgent, a
widely used large-scale multi-agent environment, by two tasks with different
map sizes. Experiments show that our method outperforms other compared methods,
and can learn effective and scalable cooperation strategies in the large-scale
multi-agent system.
