  In this paper, we propose a distributed off-policy actor critic method to
solve multi-agent reinforcement learning problems. Specifically, we assume that
all agents keep local estimates of the global optimal policy parameter and
update their local value function estimates independently. Then, we introduce
an additional consensus step to let all the agents asymptotically achieve
agreement on the global optimal policy function. The convergence analysis of
the proposed algorithm is provided and the effectiveness of the proposed
algorithm is validated using a distributed resource allocation example.
Compared to relevant distributed actor critic methods, here the agents do not
share information about their local tasks, but instead they coordinate to
estimate the global policy function.
