  Self-play, characterized by agents' interactions with copies or past versions
of itself, has recently gained prominence in reinforcement learning. This paper
first clarifies the preliminaries of self-play, including the multi-agent
reinforcement learning framework and basic game theory concepts. Then it
provides a unified framework and classifies existing self-play algorithms
within this framework. Moreover, the paper bridges the gap between the
algorithms and their practical implications by illustrating the role of
self-play in different scenarios. Finally, the survey highlights open
challenges and future research directions in self-play. This paper is an
essential guide map for understanding the multifaceted landscape of self-play
in RL.
