  Measuring the contribution of individual agents is challenging in cooperative
multi-agent reinforcement learning (MARL). In cooperative MARL, team
performance is typically inferred from a single shared global reward. Arguably,
among the best current approaches to effectively measure individual agent
contributions is to use Shapley values. However, calculating these values is
expensive as the computational complexity grows exponentially with respect to
the number of agents. In this paper, we adapt difference rewards into an
efficient method for quantifying the contribution of individual agents,
referred to as Agent Importance, offering a linear computational complexity
relative to the number of agents. We show empirically that the computed values
are strongly correlated with the true Shapley values, as well as the true
underlying individual agent rewards, used as the ground truth in environments
where these are available. We demonstrate how Agent Importance can be used to
help study MARL systems by diagnosing algorithmic failures discovered in prior
MARL benchmarking work. Our analysis illustrates Agent Importance as a valuable
explainability component for future MARL benchmarks.
