  Communication is supposed to improve multi-agent collaboration and overall
performance in cooperative Multi-agent reinforcement learning (MARL). However,
such improvements are prevalently limited in practice since most existing
communication schemes ignore communication overheads (e.g., communication
delays). In this paper, we demonstrate that ignoring communication delays has
detrimental effects on collaborations, especially in delay-sensitive tasks such
as autonomous driving. To mitigate this impact, we design a delay-aware
multi-agent communication model (DACOM) to adapt communication to delays.
Specifically, DACOM introduces a component, TimeNet, that is responsible for
adjusting the waiting time of an agent to receive messages from other agents
such that the uncertainty associated with delay can be addressed. Our
experiments reveal that DACOM has a non-negligible performance improvement over
other mechanisms by making a better trade-off between the benefits of
communication and the costs of waiting for messages.
