  In this paper, we explore using deep reinforcement learning for problems with
multiple agents. Most existing methods for deep multi-agent reinforcement
learning consider only a small number of agents. When the number of agents
increases, the dimensionality of the input and control spaces increase as well,
and these methods do not scale well. To address this, we propose casting the
multi-agent reinforcement learning problem as a distributed optimization
problem. Our algorithm assumes that for multi-agent settings, policies of
individual agents in a given population live close to each other in parameter
space and can be approximated by a single policy. With this simple assumption,
we show our algorithm to be extremely effective for reinforcement learning in
multi-agent settings. We demonstrate its effectiveness against existing
comparable approaches on co-operative and competitive tasks.
