  Almost all multi-agent reinforcement learning algorithms without
communication follow the principle of centralized training with decentralized
execution. During centralized training, agents can be guided by the same
signals, such as the global state. During decentralized execution, however,
agents lack the shared signal. Inspired by viewpoint invariance and contrastive
learning, we propose consensus learning for cooperative multi-agent
reinforcement learning in this paper. Although based on local observations,
different agents can infer the same consensus in discrete space. During
decentralized execution, we feed the inferred consensus as an explicit input to
the network of agents, thereby developing their spirit of cooperation. Our
proposed method can be extended to various multi-agent reinforcement learning
algorithms with small model changes. Moreover, we carry out them on some fully
cooperative tasks and get convincing results.
