  Traditionally, Deep Artificial Neural Networks (DNN's) are trained through
gradient descent. Recent research shows that Deep Neuroevolution (DNE) is also
capable of evolving multi-million-parameter DNN's, which proved to be
particularly useful in the field of Reinforcement Learning (RL). This is mainly
due to its excellent scalability and simplicity compared to the traditional
MDP-based RL methods. So far, DNE has only been applied to complex single-agent
problems. As evolutionary methods are a natural choice for multi-agent
problems, the question arises whether DNE can also be applied in a complex
multi-agent setting. In this paper, we describe and validate a new approach
based on Coevolution. To validate our approach, we benchmark two Deep
Coevolutionary Algorithms on a range of multi-agent Atari games and compare our
results against the results of Ape-X DQN. Our results show that these Deep
Coevolutionary algorithms (1) can be successfully trained to play various
games, (2) outperform Ape-X DQN in some of them, and therefore (3) show that
Coevolution can be a viable approach to solving complex multi-agent
decision-making problems.
