  This paper studies a class of multi-agent reinforcement learning (MARL)
problems where the reward that an agent receives depends on the states of other
agents, but the next state only depends on the agent's own current state and
action. We name it REC-MARL standing for REward-Coupled Multi-Agent
Reinforcement Learning. REC-MARL has a range of important applications such as
real-time access control and distributed power control in wireless networks.
This paper presents a distributed policy gradient algorithm for REC-MARL. The
proposed algorithm is distributed in two aspects: (i) the learned policy is a
distributed policy that maps a local state of an agent to its local action and
(ii) the learning/training is distributed, during which each agent updates its
policy based on its own and neighbors' information. The learned algorithm
achieves a stationary policy and its iterative complexity bounds depend on the
dimension of local states and actions. The experimental results of our
algorithm for the real-time access control and power control in wireless
networks show that our policy significantly outperforms the state-of-the-art
algorithms and well-known benchmarks.
