  Cooperative multi-agent policy gradient (MAPG) algorithms have recently
attracted wide attention and are regarded as a general scheme for the
multi-agent system. Credit assignment plays an important role in MAPG and can
induce cooperation among multiple agents. However, most MAPG algorithms cannot
achieve good credit assignment because of the game-theoretic pathology known as
\textit{centralized-decentralized mismatch}. To address this issue, this paper
presents a novel method, \textit{\underline{M}ulti-\underline{A}gent
\underline{P}olarization \underline{P}olicy \underline{G}radient} (MAPPG).
MAPPG takes a simple but efficient polarization function to transform the
optimal consistency of joint and individual actions into easily realized
constraints, thus enabling efficient credit assignment in MAPG. Theoretically,
we prove that individual policies of MAPPG can converge to the global optimum.
Empirically, we evaluate MAPPG on the well-known matrix game and differential
game, and verify that MAPPG can converge to the global optimum for both
discrete and continuous action spaces. We also evaluate MAPPG on a set of
StarCraft II micromanagement tasks and demonstrate that MAPPG outperforms the
state-of-the-art MAPG algorithms.
