  Multi-agent reinforcement learning (MARL) provides an efficient way for
simultaneously learning policies for multiple agents interacting with each
other. However, in scenarios requiring complex interactions, existing
algorithms can suffer from an inability to accurately anticipate the influence
of self-actions on other agents. Incorporating an ability to reason about other
agents' potential responses can allow an agent to formulate more effective
strategies. This paper adopts a recursive reasoning model in a
centralized-training-decentralized-execution framework to help learning agents
better cooperate with or compete against others. The proposed algorithm,
referred to as the Recursive Reasoning Graph (R2G), shows state-of-the-art
performance on multiple multi-agent particle and robotics games.
