  Learning to collaborate has witnessed significant progress in multi-agent
reinforcement learning (MARL). However, promoting coordination among agents and
enhancing exploration capabilities remain challenges. In multi-agent
environments, interactions between agents are limited in specific situations.
Effective collaboration between agents thus requires a nuanced understanding of
when and how agents' actions influence others. To this end, in this paper, we
propose a novel MARL algorithm named Situation-Dependent Causal Influence-Based
Cooperative Multi-agent Reinforcement Learning (SCIC), which incorporates a
novel Intrinsic reward mechanism based on a new cooperation criterion measured
by situation-dependent causal influence among agents. Our approach aims to
detect inter-agent causal influences in specific situations based on the
criterion using causal intervention and conditional mutual information. This
effectively assists agents in exploring states that can positively impact other
agents, thus promoting cooperation between agents. The resulting update links
coordinated exploration and intrinsic reward distribution, which enhance
overall collaboration and performance. Experimental results on various MARL
benchmarks demonstrate the superiority of our method compared to
state-of-the-art approaches.
