  We propose deep distributed recurrent Q-networks (DDRQN), which enable teams
of agents to learn to solve communication-based coordination tasks. In these
tasks, the agents are not given any pre-designed communication protocol.
Therefore, in order to successfully communicate, they must first automatically
develop and agree upon their own communication protocol. We present empirical
results on two multi-agent learning problems based on well-known riddles,
demonstrating that DDRQN can successfully solve such tasks and discover elegant
communication protocols to do so. To our knowledge, this is the first time deep
reinforcement learning has succeeded in learning communication protocols. In
addition, we present ablation experiments that confirm that each of the main
components of the DDRQN architecture are critical to its success.
