  We study a multi-agent reinforcement learning (MARL) problem where the agents
interact over a given network. The goal of the agents is to cooperatively
maximize the average of their entropy-regularized long-term rewards. To
overcome the curse of dimensionality and to reduce communication, we propose a
Localized Policy Iteration (LPI) algorithm that provably learns a
near-globally-optimal policy using only local information. In particular, we
show that, despite restricting each agent's attention to only its $\kappa$-hop
neighborhood, the agents are able to learn a policy with an optimality gap that
decays polynomially in $\kappa$. In addition, we show the finite-sample
convergence of LPI to the global optimal policy, which explicitly captures the
trade-off between optimality and computational complexity in choosing $\kappa$.
Numerical simulations demonstrate the effectiveness of LPI.
