  Multi-agent Reinforcement Learning (MARL) has gained wide attention in recent
years and has made progress in various fields. Specifically, cooperative MARL
focuses on training a team of agents to cooperatively achieve tasks that are
difficult for a single agent to handle. It has shown great potential in
applications such as path planning, autonomous driving, active voltage control,
and dynamic algorithm configuration. One of the research focuses in the field
of cooperative MARL is how to improve the coordination efficiency of the
system, while research work has mainly been conducted in simple, static, and
closed environment settings. To promote the application of artificial
intelligence in real-world, some research has begun to explore multi-agent
coordination in open environments. These works have made progress in exploring
and researching the environments where important factors might change. However,
the mainstream work still lacks a comprehensive review of the research
direction. In this paper, starting from the concept of reinforcement learning,
we subsequently introduce multi-agent systems (MAS), cooperative MARL, typical
methods, and test environments. Then, we summarize the research work of
cooperative MARL from closed to open environments, extract multiple research
directions, and introduce typical works. Finally, we summarize the strengths
and weaknesses of the current research, and look forward to the future
development direction and research problems in cooperative MARL in open
environments.
