  Learning communication strategies in cooperative multi-agent reinforcement
learning (MARL) has recently attracted intensive attention. Early studies
typically assumed a fully-connected communication topology among agents, which
induces high communication costs and may not be feasible. Some recent works
have developed adaptive communication strategies to reduce communication
overhead, but these methods cannot effectively obtain valuable information from
agents that are beyond the communication range. In this paper, we consider a
realistic communication model where each agent has a limited communication
range, and the communication topology dynamically changes. To facilitate
effective agent communication, we propose a novel communication protocol called
Adaptively Controlled Two-Hop Communication (AC2C). After an initial local
communication round, AC2C employs an adaptive two-hop communication strategy to
enable long-range information exchange among agents to boost performance, which
is implemented by a communication controller. This controller determines
whether each agent should ask for two-hop messages and thus helps to reduce the
communication overhead during distributed execution. We evaluate AC2C on three
cooperative multi-agent tasks, and the experimental results show that it
outperforms relevant baselines with lower communication costs.
